{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mnist手写数字识别 - 优化版\n",
    "\n",
    "这个作业中，我们使用tensorflow来实现一个简单的手写数字识别的网络，并用这个网络来做个 简单的识别示例。\n",
    "本作业中，需要参与者应用视频中学到的知识：dropout，learingratedecay，初始化等等，将网络最终在validation数据上的得分尽可能的提高。\n",
    "\n",
    "## 基础作业\n",
    "\n",
    "### 问题描述\n",
    "运⾏课上给出的notebook的代码： \n",
    "https://www.tinymind.com/ai100/notebooks/74 \n",
    "给出代码的运⾏log截图并提供⼼得体会⽂档描述对整个模型构建及训练过程的理解。 \n",
    "\n",
    "### 批改标准\n",
    "- 代码不作为评判标准，如果运⾏正确，则认为代码没有错误。 \n",
    "- 没有明显报错的正常的log输出 60分。 \n",
    "- 对模型结构的理解10分。 \n",
    "- 对模型训练过程（梯度如何计算，参数如何更新）的理解10分。 \n",
    "- 对计算图的理解10分。 \n",
    "- 解释这⾥的模型为什么效果⽐较差10分。\n",
    "\n",
    "## 进阶作业\n",
    "\n",
    "### 问题描述\n",
    "以课上给出的代码为基础，通过适当的改造，修改初始化⽅式，增加正则化，调整神经元个数，增加隐层等，将这 \n",
    "个模型的验证集validation准确率提⾼到98%以上。 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 导入用到的库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "# 设置日志打印级别为info\n",
    "tf.logging.set_verbosity(tf.logging.INFO)\n",
    "\n",
    "# 关闭ssl验证\n",
    "import ssl\n",
    "ssl._create_default_https_context = ssl._create_unverified_context"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 导入数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-2-017571120aea>:2: read_data_sets (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:260: maybe_download (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please write your own downloading logic.\n",
      "WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:262: extract_images (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ./train-images-idx3-ubyte.gz\n",
      "WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:267: extract_labels (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use tf.data to implement this functionality.\n",
      "Extracting ./train-labels-idx1-ubyte.gz\n",
      "Extracting ./t10k-images-idx3-ubyte.gz\n",
      "Extracting ./t10k-labels-idx1-ubyte.gz\n",
      "WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/contrib/learn/python/learn/datasets/mnist.py:290: DataSet.__init__ (from tensorflow.contrib.learn.python.learn.datasets.mnist) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use alternatives such as official/mnist/dataset.py from tensorflow/models.\n",
      "(55000, 784)\n",
      "(55000,)\n",
      "(5000, 784)\n",
      "(5000,)\n",
      "(10000, 784)\n",
      "(10000,)\n"
     ]
    }
   ],
   "source": [
    "# 导入数据集\n",
    "mnist = input_data.read_data_sets('./')\n",
    "\n",
    "# 查看每个数据集的shape\n",
    "# 训练集的images和lables的shape\n",
    "print(mnist.train.images.shape)\n",
    "print(mnist.train.labels.shape)\n",
    "# 验证集的images和lables的shape\n",
    "print(mnist.validation.images.shape)\n",
    "print(mnist.validation.labels.shape)\n",
    "# 测试集的images和lables的shape\n",
    "print(mnist.test.images.shape)\n",
    "print(mnist.test.labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从上面结果可以看出：\n",
    "- 训练集图片有55000个，每个数据是一个28*28=784的一维向量\n",
    "- 训练集的labels也有55000个数据，对应着上面每一个图片的值\n",
    "- 验证集的图片有5000个，每个数据也是一个784的一维向量\n",
    "- 验证集的labels也是5000个，对应着验证集每一个图片的值\n",
    "- 测试集图片有10000个，每个数据也是一个784的一维向量\n",
    "- 测试集的labels也是10000个，对应着测试集每一个图片的值\n",
    "由此看出，数据是完整的，没有缺失\n",
    "\n",
    "可以看到images里面有数量不等的图片，每张图片是28*28长度的一个一维向量， 所以用的时候需要先给它还原成28*28的二维图片。labels中则是图片对应的数字的值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 查看数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.38  0.376 0.302 0.463 0.239 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.353 0.541 0.922 0.922 0.922 0.922 0.922 0.922 0.984 0.984 0.973 0.996 0.961 0.922 0.745 0.082 0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.549 0.984 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.741 0.09  0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.886 0.996 0.816 0.78  0.78  0.78  0.78  0.545 0.239 0.239 0.239 0.239 0.239 0.502 0.871 0.996 0.996 0.741 0.082 0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.149 0.322 0.051 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.133 0.835 0.996 0.996 0.451 0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.329 0.996 0.996 0.918 0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.329 0.996 0.996 0.918 0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.416 0.616 0.996 0.996 0.953 0.2   0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.098 0.459 0.894 0.894 0.894 0.992 0.996 0.996 0.996 0.996 0.941 0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.267 0.467 0.863 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.996 0.557 0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.145 0.733 0.992 0.996 0.996 0.996 0.875 0.808 0.808 0.294 0.267 0.843 0.996 0.996 0.459 0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.443 0.859 0.996 0.949 0.89  0.451 0.349 0.122 0.    0.    0.    0.    0.784 0.996 0.945 0.161 0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.663 0.996 0.69  0.243 0.    0.    0.    0.    0.    0.    0.    0.188 0.906 0.996 0.918 0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.071 0.486 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.329 0.996 0.996 0.651 0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.545 0.996 0.933 0.224 0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.824 0.98  0.996 0.659 0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.949 0.996 0.937 0.224 0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.349 0.984 0.945 0.337 0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.02  0.808 0.965 0.616 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.016 0.459 0.271 0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]\n",
      " [0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.    0.   ]]\n"
     ]
    }
   ],
   "source": [
    "# 格式化打印\n",
    "np.set_printoptions(precision=3, linewidth=256)\n",
    "# 打印第一个图片的二维数组\n",
    "print(mnist.train.images[0].reshape(28, 28))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由上图可以看出mnist里的图片像素值都是0-1之间的连续值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 576x576 with 0 Axes>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x128363f98>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[7]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1311eb860>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x13320d160>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[3]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x13321def0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x133235710>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[4]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1332507b8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1332690f0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[6]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x133280080>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x13328f828>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[1]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1332a7908>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1332c1278>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[8]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1332d5080>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1332e59b0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[1]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x13331a128>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x13331a320>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[0]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x13332e320>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x13331a278>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[9]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x13336d390>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x13336d5f8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[8]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x133386400>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x133395cf8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[0]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1333afc88>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1333c7518>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[3]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1333df550>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1333eef98>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[1]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x133420470>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x133420908>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[2]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x133448ef0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x133448dd8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[7]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1334775c0>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x133477978>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '[0]')"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1334916d8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAHiCAYAAACtERYWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd81EX6wPFnUqgJXVF6DU0pKvaCAhbE3hvqoSjYOBX19Px5nGc5KyA2wIIV26Fgwwo2FESUIr1IB+k9JNn5/ZEw8501Gzab3dmUz/v18nXPZGZ35/hm8+x3ZmdGaa0FAAAkXkqyOwAAQEVB0gUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADwh6QIA4AlJN0AppZVSO5RSD0TZvq9SanvB41olun8oHq5n+RLD9Rxc0F4rpdIS3T8UX0V8jyo2x7CUUlpEWmutFxaUjxORT8KaVReR87XW70V6HEqHQq5nPRH5QETaikiqiMwRkdu11t8X9TiUDkVdF6VUHxEZLSLXaq1HBX7eTESWiEi61jrXU1cRpcKuqVIqVUQGi8jfRCRTRBaKyIla681FPa6s4NNfEbTW34pIxt6yUqqbiIwXkU+T1SeUyHbJfyMvEBEtImeJyHil1P78QS67lFK1ReRuEZmd7L4gLgaLyNEicpSILBORDiKyO6k9iiOGl4vnShF5V2u9I9kdQfFprXdrredprUMiokQkT0Rqi0id5PYMJfSQiAwTkfXJ7ghKpuAD1EDJH7H4Q+ebpbUm6VY0SqnqInK+5A9hoQxTSs2Q/E/O40RklNZ6XZK7hBgppQ4XkcNE5Llk9wVxcbCI5IrI+UqpNUqp+UqpG5LdqXhieDl650r+J+lJye4ISkZr3VEpVUVEzhGRSsnuD2JTMPf3jIjcqLUOKaWS3SWUXCMRqSkiWSLSXERai8iXSqn5WuvPk9qzOOFON3pXisgrmm+elQsFQ81vishdSqlOye4PYjJARGZorX9MdkcQN7sK/vffWutdWusZIjJGRHolsU9xRdKNglKqsYh0E5FXktwVxF+6iLRIdicQk+4ick7BMOQayf/yzeNKqeFJ7hdiN6Pgf4M3N+XqRofh5ehcISI/aK0XJbsjiJ1S6kjJ/52fIvlLhm4Wkfoi8lMy+4WYXSUiVQLl/4nIuyLyQlJ6gxLTWi9SSn0rIvcopW6W/A/EF4vIJcntWfyQdKPTR0QeTXYnUGKVJf9bri1EJEdEZorI6VrrVUntFWISXLcpIqKU2iMiW7XWW5LUJcTHJZL/wWmDiKwTkXu11l8mt0vxw/CyK1tEpiml7g/+UGvdVmv9l0/PSqmrlVKbCx4X8tRHRM+5nlrrSVrrTlrrTK11Ha31CVrrb/Y25nqWeoW+P/fSWncL2xjjPhH5reBx5WqIshz5yzXVWq/UWp+qtc7QWrfQWj+/t648vEfZkQoAAE+40wUAwBOSLgAAnpB0AQDwxOu3l3umXMAEcpJ8Hnon7tv1cD2TJxHXU4Rrmky8R8uXSNeTO10AADwh6QIA4AlJFwAAT0i6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJxztBwAo/VJSTTh/ZBenavYpz5j4jCv7mzjty2mJ71cxcacLAIAnJF0AADwh6QIA4AlzugCAUietaWOnPP+huiZe0m1UWOtKJtrc0sb1vkxI10qEO10AADwh6QIA4AnDy6gwUttnmXhu/9pO3YJznzVxSNwjSFPEHov5zObmJh79RC+nXd0XJseln0BFldaimYl/v6eeU/fXIWXr2uXHmLj+t+tNnBe/rsUNd7oAAHhC0gUAwBOGl1GupDVu5JR/v+8AE7950vMm7lI55LQLBT5/hsStC3427VdroYkb3Pm60+rFCceZOHfFyug7jb9IqVLFxE2+UU7dMw2/N3Gqstdmzp6dTrvbTulj4rx5CwWlk0q33zae8686Jl7SI/Jwcosv/uaU2/T73cSh3Qvi2Lv4404XAABPSLoAAHhC0gUAwJMKP6e7+tajnbIKrBapssEWNrV1H3fgZPtl9CrjpySkb4jO4keOMvHcy5526oLLf4JLf0Jhnzc/2lnTxFO2t4j4WodWX2ri8zK2OnWrJswy8Ycd3CVJ2LfgPO7KMXZp1ocNXy+suYiIdJt1tonV4+4Sk8qLfi1xn9KaNTFx7tJlJX4+/NW84Z1MvKTHyIjtWk28ysSt+/zi1IV/C6M0404XAABPSLoAAHhSKoeX193gDvlu7phj4rEnD4/ra7WrNDVi3W6da+KaKVWdunVX7DDxqmHuP+MTa3qaeMOFNUycu3xFzP1EZBf0tEtIwneTcpf/2M+YT29u6bT7/JQOJi5quc/3Z1xs4jOfe9apCy4n+lC6Ft1p/MXCf9mDyed2fTpiu9ZfXmPiNv3nmTi0Y6nTzv1NiM78Ee51++Dkp0x80cu3OnVN/vVDDK+AhUOOdMu9nwmU7Hu0xefusqCsfrNNHMu1LS240wUAwBOSLgAAnpB0AQDwpNTM6c4faedS5vYa6tRVVunBkqcehb+ua//U6oHYrXul6TcmvvytbibedGkTpx1LEErg8INNeH1dO7f60c4DnGbB5T+ztjYwcfag/Zx2ix6xFzHr/mpOXd4cu61ccHlY+vPuhc8JTDStvNP9XkLD/zL/F04f1ckpf3Ppo4GSvQbLct3tHbP62qVZoZw9Je5HTo9DTTy2p/udkQ6BLQoRuz2n2r/vY88e4tSlKrtUzFkWdPVvTjsdKo1nBhUfd7oAAHhC0gUAwJNSM7z87ImvmDh8WPe/G1qbeN2ezJie/3/T7BBSk/GqiJbRWdHdfl55pNcbTl1wp6LXmk008eVvdHPabbrInojDcqJimjLThP3O62/i1NUbnWbu8p81Jlp5p7vr1JwT7NKQ00Ze69SlzrHxhr5296scPc1pF1ye1PT1P9x+hPcfsvZOd2h4/1Q7pLxL27o+A29z2lXL+Smu/dj+d/t+PbiS+7dnu842cfN3Njh15WOw04+69ywxccdKVZy6nnPOMHHWffZa5JWT4eRw3OkCAOAJSRcAAE9KzfDykIvON/E/O9dw6vZ/3+46k7fBHT6MVpZE3nkqFq3G23jUi72cujVj7A5JN9RabuLgULOISJt+dli02b0ML8dKT7VDzdEO41ZZ7+5pM2JLMxNXWrvdqVs82H4T+eUr7DB08AAFEZFp2fYzLIfY71u/rO8i1p0z7wITVxsbeThZpdk/Yapq1YjtwuUdbKcXnmz3UsR23aZdbeL9Z8+N+vnhurnhFxHrto6202y1Fkz20Z2k4k4XAABPSLoAAHhC0gUAwJNSM6erp9kTJOq6KzFK/VfzQzPcuZ6Xnuxt4hsGPxve3Hjjcrvz1t33Hh7/jlVAu85y/x03trW/4sF53Loz3XnbfjWXmrjzh+5yn8Mr28cFlwVNzXY/s/6zr11qlCruIdsonsz03SbeEVaXc/JhJq5z71ITv9Xis2K8wqRCf/p92DXd72F/O+CVN1sut6cJHV/lVxMfM+Ncp12tV3/01qfSgDtdAAA8IekCAOBJqRleBuJh1UXuLkdzTrDD+8ElPuGH3QfrgsPJ4XXBZUFXvHuj067F1+V/uUM8jRh5hlO+/nZ72MArLeyavOt/ONVp90JTe03TJOy0kRK6avz1Trn15Io19BlPm88MnxjIt3OceyhJhl6cuE6khP1+lIJdrrjTBQDAE5IuAACeMLwcByvuds9ODXXZFtXj6qfaodDckw516tK+mhbeHDEIfts4+BnT/XnRdf2Wn2Ti5f+wh28wnFwyOxqFXwOrqrLn2I5u+lVYrR0yvG2N/bb6xxO6Oq1yDrTvr4Unj4yqT/V+KflhKMh3YO2thf686obI1z1W2afZa7/+Wnv+8kH1Vzvttp1vf69yV6+RZOBOFwAAT0i6AAB4QtIFAMCTCj+nm9aimVNe2PdAEz9z8YionqNbFXf3oVQV3WeZRmkZJh7x0lCnbkDTY6N6DrgavFXJKV/Q0C5LOajGKhNfX/cHp13DwAHq4Z9FFz3UzsRVv54Sh15CRCTr+T+dcrucG6J6XKtX7UljoXmLTNw8151jX/zwUVE934CVx5i4zhvudyl0eGNElHZAfac8ss3rgVKGlFRqrZomPnvyAqfuosxhJq6ZEvm0qQ7DLzNxo/OY0wUAoFwj6QIA4EmFGV7efsERJv7zEPtZ49/njnHaXZy5KYZnL/lnlx5fDHTKWfJziZ+zIqr6gTv8m/2BjacFrlO/rv2ddtvut7vnfHXwW07dsf+yuxL9Nq2xiTmovmTy5i9yys3vWhShZdjjonz+tJ3RLf/5eVRnE9fLYRlYzNLTnWKTtJINKa8b4C7FPPu6iSbuV3NVWOvIQ8pB+2UWvkuWT9zpAgDgCUkXAABPSLoAAHhSruZ0VZcOJq413N3+6+Nm9mSSaJf0vL/DzknM2tUoYrsPH+nmlFOz7UKDK/9tT0v56zyEVWlNesS6iiKtsftvnLt8RcJeS0+d6ZQzAgfZXDDJPf1mbKuPTXzQNXYpV5N/MadbmqkiJn9zAzPDtedne+hN+ae3udvfjtjSwMRF/e1LrVfXxMv/1sbEMwc+E8fe5duyq4qJ94/7s0eHO10AADwh6QIA4EmZHl7+Y7D7lfJ7L7ZLPS7L3ODULcu1J0/M3VPbxDe9eY3Trtpqu8zgwInrTZz3+/yI/agpkQ+6XvCPwC4tYUMsS3K2m7jZB9ulItp1lj0lJrg0R0Tkwz/sdMGBZ8/x1qctjzVxyqHn7HRBTutd3vqBkrn6kgkR6y5YaKcQUif+ErEdope3eYtTfnOFPfmnX027du+YO39y2nW93x5if2HGl3Ht0+A/2zvlBjfbJUO5cX2l6HGnCwCAJyRdAAA8KdPDy7W6rnPKwSHl7r+f6dTlPHWAiYO7FjWTyDvQRLvzTbjQCV1MfHatFwI17mecjaHA5vxT3G/TlmfBbylf9NAnJv55azOnnc8h5eBm6uc/7A5LpggHm5cFqfvt55RbV14Yse36Z5uZOFOSs/F9ebf7JXt4TPajOSZ+9IDpcX+tHG3/Wref1NfEWf9wpxlz/1ge99cuLu50AQDwhKQLAIAnJF0AADwp03O6dfu6y2xa3WpPjmk5yJ2rTZNlXvokIrIpy+56ckyVyJ9r+s263MT1JPKSpPLmj0vtkpzgUoInp/dw2rWU+M/9GIcf7BRPe+kb26da7lxgKPDZNH1+dKeZwL8tJ7Z0ymdUs3Pz27W761SV9TmCxKrxhl0C+NN/7I57x1cprPW+5emQiQ/7+VKnrtK7dhloi1ft3/5kLQsqCne6AAB4QtIFAMCTMj28nLva/ap/y0Gl46v/G7oWPqgxZ89Op5z5TM1C25V3Db+2G6On35Jq4ls6f+W0e+Gm001cd7Y7PJj21bRCnzu1fZZTXtW9nokzTre/H18f/LLTLrgsKBT2WTTrk+tsPPiHQl8XyXfl4HER65bkuNc0/YvCf3/gX9vvrjCxmpVp4ubDZjvtdJ4dXt5/29zEdyxBuNMFAMATki4AAJ6QdAEA8KRMz+mWFqfM2uqUx9Z6OlCyWz1eOftKp13tT6YmslulV2DLy2NmnGvirw5+y2l2/V1PmTgkIadu8LpDC33qM2u+6ZS7VLaPSwl8xgx/vuDnzzbv3uDUtH/Ubh1XGpcgIF/d1MgndT22+pSwn2xObGcQUftnBzjlZg/ZbXl1rn2HxboNb2nHnS4AAJ6QdAEA8ITh5Tg4v8YMp1wtJcPE83PsocnVhtfy1qeyota1e0w8eJw7ZPxgffvvmqOdKrl//19NHBJbGX4iUHD5z9o8ewD9MxuOdtp9NvwYE7d+wd3NjCHlsm9PKHXfjZAwD7TobOLG4i670+GNyznudAEA8ISkCwCAJwwvx2jdADs8WT/V/Rbykhz7LcpLHhxk4nqfuMOWEMldvsLEv53R2Klr9d/Cv6EsIjKn2ygTHz/jQhP/ubFGxMe0GmIHivXUmU5dXeHalGcjm33olA99/O8mbnnbj+HNgYThThcAAE9IugAAeELSBQDAE+Z0o6QqV3bK511vT8TZFtrj1PWa0t/ETZ5nrjBauStWOuWWl62M0FKkt9j53hqyKBBHVtGWJlQ094y5zCm37fOEjdPd96+E3KVlgC/c6QIA4AlJFwAATxhejlbIHZx8dfyJJv7kt25OXZO3WYIA+Nb0/9ypnFv/76iIbVuyRAxJwp0uAACekHQBAPCEpAsAgCfM6UZJ57jLgprdw5wQAKB4uNMFAMATki4AAJ4ordmnBwAAH7jTBQDAE5IuAACekHQBAPCEpAsAgCck3QCllFZK7VBKPRBl+75Kqe0Fj2uV6P6heGK4nj0KrmdIKdUj0f1D8fD+LH9iuKaDC9prpVSZ3GeCpPtXnbTW9+wtKKVGKKXmFfwhvirYUGv9gtY6w3sPURzh1/MkpdQvSqmtSqnFSql+e+u01l8UXM9lSekposH7s/wJv6adlVLTlFI7C/638946rfV9ItIhKb2ME5Luvv0mIgNE5JdkdwQlo5RKF5GxIvK8iNQUkYtE5AmlVKekdgwlwfuzHFFKVRKRD0TkNRGpLSKjReSDgp+XCyTdfdBaP621/lJEdie7LyixOiJSQ0Re1fmmisgcEWmf3G4hVrw/y51ukr898RCtdbbWepiIKBE5Kam9iiOSLioMrfVaEXlTRK5WSqUqpY4SkaYi8l1yewagQAcRmaHdXZtmSBkfUg4qkxPRQAm8KSKjRGRoQbm/1np5EvsDwMoQkS1hP9siIplJ6EtCcKeLCkMp1VZExohIHxGpJPmfnu9QSp2e1I4B2Gu75E8BBdUQkW1J6EtCkHRRkRwkIvO11hO01iGt9TwR+UhETktyvwDkmy0iHZVSKvCzjgU/LxdIuvuglKqklKoi+ZP56UqpKkop/t3Kpuki0rpg2ZBSSrUUkd6SP2eEMoj3Z7kzUUTyRORmpVRlpdSNBT//Knldii9+OfftMxHZJSJHi8iIgvj4pPYIMdFaLxKRv4nIMBHZKiKTROQ9yZ/jRdnE+7Mc0VrvEZGzJX8KaLPkv1/PLvh5uUDSdWWLyDSl1P17f6C17qa1VmH/TRQRUUpdrZTaXPC4UHK6jCIUdj3f1lofpLXO1Fo30lrfqbUOiYgopboXXM/6kv9pG6UL78/yp7BrOl1rfajWuqrW+hCt9fS9dUqp+yR/bXa2iJTJc2k5TxcAAE+40wUAwBOSLgAAnnjdHKNnygWMZSfJ56F31L5bFQ/XM3kScT1FuKbJxHu0fIl0PbnTBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgCUkXAABPSLoAAHhC0gUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADzxespQWbZkTEen/N0xz5r40j43OXWpX//ipU8AIlv0+JEmvuXUT5y6jy85ysShGXO99Qn7cKT9O7vkFveQnvknjDZxq4lXOXUtL/01od2KJ+50AQDwhKQLAIAnDC9HSS+r7pTrHlfVxBvbVHbq9vvaS5cQR9mndzXxxmu3O3XTu74e1XNcv+I4E3/3SSenrsXzi02cu3pNLF3EPqQ1bOCUh5/1kol7Vt3l1I0+opeJ685IbL9QtDUDjzbxgze+aOKTq+5w2uVoGw89fIxTN0zaFvrca2862ik3eMNOJeRt2FjsvsYDd7oAAHhC0gUAwBOGl6NUfYWKWHfARX845bznEt0bxEKlVzLx/Ce6OHUfnfGkiVulu9MFoSif/7lG39rHXPuNU9f54D4mbnQew8uJsOi6pk45fEgZyaMq2/fUpgsPceq+uf1xE1dTlaSkVvzDDilPvWGIU/f2DY1MPGzIeU7dfs9NLvFrR4M7XQAAPCHpAgDgCUkXAABPmNONg1256U655LMSSIR5T3U28fwznnHqUqSKiUOiJRr9lndzyqMaT4rYdlhnu8Th8bonmDhZyxbKo8bHrEh2FxDB4n/ZedzZfYaH1Ub3F/O5zS1M/Pyrpzt1DeUHE2fXtd/CSFepTrvLMlebuOtdTzh1V8itJk7k/C53ugAAeELSBQDAE4aXo1Tj9NUR67a85+6Es5/8EaElEi24LEjEHVKe3Ts4rOUOO63O22ni48fe7tS1GLvHxJUX2OU+ees3OO26vHWZiad1fc2p+2VXMxPrPTkReo/i2t37cBMPbfFUWG26IHmCy4Sqt99U7Md/sjPTKb93x8kmbvjRD+HNiy0r7G/FmH88ZuJTugy07a6bWuLXCuJOFwAAT0i6AAB4QtIFAMAT5nSLkNfNfs19fIennbpf99g5wfqvz3Lqot02EPG3+obDnPL8M4LzfPaavbClidPuf9f2NHHr73+M+Py5Rbx2dnbkOcTxK+3h3FW3LSniWVAcu+raa3pwJeZwk0mluelk0b/t38/fDwtfJlS44DK8dee5c7qVV0Y3t9rsI/sdjI5Nr3Lqph31gonDlxM1T7PLBmvMTdzvEne6AAB4QtIFAMAThpeLkFfZfibJUO7JMzna7loU2rbNW59QtP79PnDKKWJPh3poQ3sTTz4zy2mnlv4a1fOn1qhh4hXXHOTU3dHxfyaevsedZKh6CkPKyfR9tnt/kbm8qIkCxCK7h3ty1++XRzekfMuqY0y89nQ7rJu3YVVM/Uj9+hcTN/narRs770ATX5ixLqbnLynudAEA8ISkCwCAJwwvF2HpOXwmKWvywj5HBg8v+PjBbibOXBr5G8qS4n6rMe+ETibuPfxLE19fyx27Cg5lnz7v7LAnXRn59RCzttfPjqrdkBU9nXKlT+O7y1BFtfZme2D8gP7vR/WY4HCyiMiSE+x7NrSz/B8AQlYBAMATki4AAJ6QdAEA8IQ53SJkHsBSoPKk2po9+24k7hyuiMgnr42M6nHnLOxl4pTzdjp1eVE9A4prQP3gvLqK2G7eJ62dciP5M0E9Kt9SOrVzyg/fbHd46l51Z3hzI7jTVHBZkEhi53FVlw5OuVn6LxFaiizMyTZxzcWJW1LGnS4AAJ6QdAEA8IThZZQrC3bVd39Qc6kJX3xlmIkfXtvDaTbxj1Ym/vTwYeKqaqItod0m7vrR351WbW+zy1dCO3ZE22V40PR9dziZ4f7YHPeqOzxb1JBy0NT3DzZxww0lP4A+WvP6V3PKh1fWEVqKTNhhd6yr+sGUhPWJO10AADwh6QIA4AnDy2FSqtgzFY9tGHmT+pHrTgiUtiewRyiOOTe0d3/w3k8mPDDVDhMPbfC90yylgR3yCgWGk8Od+NQgE2c94g6TcY6yH8FdkNqkB69BFafdyrzA0GcuA8qxWn/dUSbuX/vxsFp7EMzqvF1Oza1/2F3ZmvxvrYkTfSXSmjc18aRTnwyrjfze/m5jq0BpfXw7FcCdLgAAnpB0AQDwhKQLAIAnzOmGSalV08RPNfgkYrtJ39kDzFtKESfWIOGyT+9q4uUXuzvJpBSxS1FQqgp8/tTu7Gz32eeauMEj/pY7IF9q/f2dcpdLZ5q4RkqV8OZGt7G3m7j1At6jsdpmp0glI6VyxHaPrTvRfdxxwXnRxM2Rhpt3gz2oPvg9jnCbAsv/RETWDG1p4urM6QIAUPaRdAEA8ITh5TC5zervu5GINPk0J8E9QVBKx7ZO+YAR9lD4UY2fN3Hw0Pr8cuHuWtPVKf9vymEmfrbnaKfuhTavmbjPhXbIMuNthiy9qFfbKY5q/GmhzbaGDRdmLuGewqdPvzjMKTeXyf5eXNlpJJ0a3UNuX3GaU67+7k8RWsYXv5UAAHhC0gUAwBOSLgAAnjCnG2b9PbsL/XmvuWc65UoTfzNx5HMrUBLr+9nt5ybc+5hTV9NZKhJ5WdBtq4808Sdf2TmnrCfdLT6zVttTRR478TKnLniI/cX32WVkH77tzjUiMfKqV4qq3cwc90SZA4awvMunA79P3labWy47wsRzL3w6qsf88L27ZayvpZ/c6QIA4AlJFwAATxheDvPsQa8HSva756u21nDaNchd4alHFce2i490ysEh5ZphOw/NybFLtp5c09PE84Z0cNrVfP9XE7fYbZcwuPtWuVIn/eaU2759g4l/u2CIiceefKPTLv2zn4t4VsQq8/HVUbXrP92dFmgksxPRHUTQ9O65Tnnt+Pg+f1qjhiZecEMTp+6ny4OnH0XeNevNbXZJaNZLm5w6X4Pj3OkCAOAJSRcAAE8q/PByWjN3mCJT2W88pqp0392p0NZ3dL+FHBxSHrujjlP30oWnmzj06+8mzgz7BmIsB8unVHWHsjscstTElQO/E6G06A5TQPGlNW5k4qyMZRHbXba0h4mbXrPKqePYer+OrbXQKb/f2k4X5S1YHNVzpLZrbeIFV9Zz6oac/5KJT666I+yRkYeUg0bfcJaJ02ZPi+ox8cadLgAAnpB0AQDwhKQLAIAnFX5Od/cot5yVbufz8gKHmWe87S4ZQuIFD6C/8+sLnbqsX6fG9bVS69U1cbWx7lztWy0+DpSYx/VhTa/GJh63/zinLlXZe4VNu+0uVCl73CUgKt3uZKVz9sS7ixVG61F2ydbgXp2duvv2s0vyrq6x3KlLHWf/fs7c2Uii0bn6JBNflhndUrFw43bYneJu/+Jip67tj3YZWSzf94gH7nQBAPCEpAsAgCcVcng5NauliW9rNi5iu0uW2J2Oaozxc8BxRVZvhnt0xKbQLhNP7TXEqev6/EATt/u/P0yct3ZdxOdPa9jAxDs6NXTqBg5908SnV9vi1AWHoZ7ebH93qn47N2I7JE5w2ufjtoH373y3Xet3B9j4Fj+b2ZdHuYuXmnjCsGOduoGD7b9r+K5xfWqstIVgHAc7tTtd8PRGO+z9zd+6mjjr5ylOu9LwHuVOFwAAT0i6AAB4QtIFAMCTCjmnu6dhTRN3r5odsd38t9qYuL7mQOxEyxzjzrsd32qQiX/r/5RTN7/3cyaefbI9M2jggosiPv/r7ewJUuHzT8HlSeHzPretttvZzb3JHnyttv0mSIwqG+1VWJS7y6lrmVa10MfsCpvnq7aae4p4q/PiZKf8f/27m/j6/SY6de3S47uNbvD7FK8OPc2pqzci2K9ZcX3deOO3EgAAT0i6AAB4UiGHl4ty/YrjTNzgzXkm5sQS/+q7Ywk4AAAgAElEQVTMtf/qz21u4dS1r7LCxN2q2KHhzzu8V8QzVolY89yWpiZ+8qPeTl3re6ebWO1mSNmHjHfsEr0LDxjk1P36j2dM/J/1bU383oiTnHYNhzMllGiLuu428V2tLnHrrjrAxKec+rOJHz/QnUbq8MqNJlZF/KFt+cYGE9f7fXLkhqUcd7oAAHhC0gUAwBOltd53qzjpmXKBvxeD4/PQO3HfqT+Z1zOtWRMTL3i4VsR2Dx3yvol/2NbKxOMnHOG0a3532RquSsT1FOE9mkzl7T1a0UW6ntzpAgDgCUkXAABPSLoAAHjCkiGUSblLl5m4+cXLIrYbIcGlRnaXo+ZStuZwAZQP3OkCAOAJSRcAAE9IugAAeELSBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgiddThgAAqMi40wUAwBOSLgAAnpB0AQDwhKQboJTSSqkdSqkHomzfVym1veBxrRLdPxRPDNezR8H1DCmleiS6fyieGK7n4IL2WinFiWqlUEX8m0vS/atOWut79haUUmcopWYVXOgflFLt99ZprV/QWmckp5uIUvj1PEkp9YtSaqtSarFSqt/eOq31FwXXM/JZgUi28OvZWSk1TSm1s+B/O++t01rfJyIdktJLFIe5pkqpekqp75VSG5RSm5VSk5VSx+xtWB7+5pJ0i6CUai0ir4vI9SJSS0TGi8g4PjWXTUqpdBEZKyLPi0hNEblIRJ5QSnVKascQE6VUJRH5QEReE5HaIjJaRD4o+DnKpu0i8jcR2U/yr+l/RWR8efqbS9It2iki8q3W+jutda7k/wI0FJETktstxKiOiNQQkVd1vqkiMkdE2hf9MJRS3UQkTUSGaK2ztdbDRESJyElJ7RViprXerbWep7UOSf61zJP85FsnuT2LH5LuvqmwWInIQUnqC0pAa71WRN4UkauVUqlKqaNEpKmIfJfcniFGHURkhnY3G5ghDCmXeUqpGSKyW0TGicgorfW6JHcpbki6RftCRE5QSnUrGLK6W0QqiUi15HYLJfCmiPyfiGSLyLcico/Wenlyu4QYZYjIlrCfbRGRzCT0BXGkte4o+aNSl0o5+1BM0i2C1nquiFwpIsNFZLWI1BOR30VkRTL7hdgopdqKyBgR6SP5H546iMgdSqnTk9oxxGq75P9hDqohItuS0BfEWcFQ85sicld5+t4FSXcftNbvaq0P0lrXFZH7RKSZiExNbq8Qo4NEZL7WeoLWOqS1niciH4nIaUnuF2IzW0Q6KqWCU0AdC36O8iNdRFokuxPxQtLdB6XUoQXzf/uJyAgRGVdwB4yyZ7qItC5YNqSUUi1FpLfkzwOi7Jko+V+0uVkpVVkpdWPBz79KXpdQEkqpI5VSxyqlKimlqiql7hSR+iLyU7L7Fi8k3X0bKiKbRWSeiGwSkWuT2x3ESmu9SPKXIwwTka0iMklE3hORUcnsF2Kjtd4jImdL/nTBZsm/tmcX/BxlU2UReVpENojIShHpJSKna61XJbVXccQpQwFKqd2S/wWbYVrre6Nof7WIPCkiVUSkvdZ6cYK7iGKI4Xp2l/wkXFlEemmtv05wF1EMMVzP+0TkVsm/ntW11nkJ7iKKqSL+zSXpAgDgCcPLAAB4QtIFAMATki4AAJ543US6Z8oFTCAnyeehd9S+WxUP1zN5EnE9RbimycR7tHyJdD250wUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADwh6QIA4AlJFwAAT0i6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJ14PPAB8S2va2MSbj2ho4tW99zjt+h8yycQDa8936g767moTh5ZWN3Grwb857UI7d0bux4EHmDh39Zp9dRsoV3K7H2riDR0qO3W79rdnMuhWO0x8Z6fPnHZ9a9r3zac73ecYNKKviRs88kPJOptg3OkCAOAJSRcAAE8YXka5smrQ0U75nmveNPE5GesiPi4l8PkzJCGnbsaxL9jCsTbstPsWp13T+yIPa1V+K8/EucdHbIa9lD2KdF3/o5yq/je9b+J+NVfF9PQjtjQw8ftnHmni0NIVTjud405DIHpbLrf/rl89PMzElZWbdkJS+JG/KeIeR5ujbbvuVd2pnO9uftzER6feZuJGD5W+oWbudAEA8ISkCwCAJwwvh0np1M7E826tauIrOv/ktLupzhQTd398kFN3wJDSN6RRnqW2zzJxcDhZJPKQ8p952U75j9xqJs6TdKfusEp2iDE1MOz52zVDnXZdt9rh5gMfd38Hjq2zyMQTpEahfarwUlJNuPyeI0w88/rhER+Sre2w/apc95pWCYxO7p9azanrW8MOI/ed+K6Jh25q5bT7svdBJs5duixiP/BXW8/ebuJ0Za9t+HDystxdJr5nxZkRn++nuS3s81V3h/2/O+ZZEx99tl1VsPwJ91vOOtv9HUkG7nQBAPCEpAsAgCckXQAAPKmQc7qqsh3nX9PvUKfup7vsPN22kJ03OHLM7U67bzrbuZ8TLp/q1M0bEpduIkpz78owcfgcbvAanvjztSauP7SK0y514i8Rn3/9dXbJSu8B35j47nq/Ou3y3Okjx3cbWwZKf0ZuWIGtHBTtPG6uiTu9YefRW9wx2WmX2q61ief+I9Opm3XScyYOLmG5pfZC98U+tOEX3Zo7VXnrN0TsI0SaXbvSxAM+tevkZm08wGlXO7DyLm/+IokkSzZGrDviub+beP4Zdn638203Oe0aPZj879twpwsAgCckXQAAPKkww8spVexw4twhHU288Ax3GOupzXZI6p3Bp5q45dthQ1dZdrhwRsvOTp0+w65VSNtplzSkfTmtuN1GFP533LOBkvs5csAfdglCg3N+j+n56z1vr/1X6+yWVHcP/7Ww5oWa96n9vWrE8LKIiKg0989PpWOiG6496H92yLB12JByUN6cBbZdH7fuuH52TPORO0eYuFuVHKddcLj5y8yD3SdheLlIeZs2mXj6SDtFU2uRu2wnb37kqZ1ope4o/P6xQ695TnnLgyV+qRLjThcAAE9IugAAeELSBQDAk3I7p5tSzd32beUbTU28sKtdLvDEptZOuwk3nWDijK9/jPj8wa+2V9u01akbOHmiiUetsV+V3/LlPjqNmBxcyW7bGL7F3NT5dplHlpR8Di5zlp2P/W63u+yo7uzc8OaGVhGrKqzUJo2c8tRD3yy03VObWzjlts/ZucK88MZRqjfCzgWPvfYwE3drEHmOGLGrOyo5/6696/3mlF+XRhFa+sOdLgAAnpB0AQDwpFwNLweHlOc+fpBTFxxSfmxjGxN/c2Z7p13qkuJ/fX35Ve4QdfeqE0y8cT/7fK/U6ui0y9u8pdivhb86cdZ5Jv78oLedutHdRpn4AXGXdkUrt7vdtWy/++20Qos09/rVu22JiXd84D6HKvyc7gpt6UUNItZt13ZZyZgHT3Xqav4eedonFouvambi78e7p4kdUzlk4gX93P62uNfuuKRzI08tIP6yT+vqlK/qObHQdu+v6xL2k+Qv1+NOFwAAT0i6AAB4Uq6Gl/+8rJOJF575tFP30U67Kf43Z3Uwce6SpSV+3T01I48dztlth6QYTk6MjIH21/jZd92h/n4155t4/jOHm7j9f1c77daebL/VeMaNk5y6PrXsIRgN0oKnGrgnHLzSYryJe/dyN1rPrcr4sohIat06Jr7zyrcjtnt3m/3Wec3X4zucHC5vtt216MoJ/Zy6hWfaaak5fdy/Kae/F9jm6udZielcBZZao4ZTXnuJ/bt93UB3/qZvjRUmXpq7y8QbHnUPqajC8DIAABUHSRcAAE9IugAAeFKm53TTGrpf4b9j0BsmXpm306l76L4BJq6xuORzRGktmpm492k/RW6IhAueJvPq0NOcuv732bq5ZwXm5M5ynyMl8PkzJCG3Ugo/nf7ONUc55fHf2J2N2s5c4dRd94g94WjCve5cVUWiAqd9XZa5Lok9KVyNuWF/Es8svJ2IyLzr7f+XrGsS1KFyKKWzu0xzVbdaJt7axi69uvYY97sVg+p+XcSz2i3fenx8q4mzxk+JsZeJw50uAACekHQBAPCkTA8vh+q6w3TnVbcbof97/RFOXY03ij+kHDxke+XAw526u659y8QXZyT/a+gV2a6z7LU57rqpcX/+vn/0NPGftzYxccqMhU67Vjvt7xj7E5XM15vaBkqbk9YPRC/twAOc8pWT7CEHp1RbY+J0cYd801VqiV/72Nvt9GHWW/H/GxBP3OkCAOAJSRcAAE/K9PByUc6sMd0pf9jvFhOn74y8O9DG0+1uJh8e/YyJW6a5QyLv77DfuGs17nqnLriLzdSNTQM1q4ruNKK28Wr7zeELb/vMxANrzw9rGd3nyuAQV/un3d2kGj/wQ6BkhzrDv+NclBRVnNbl1+JrmkXVbtYY+w3X+vJDES1RWuja7nTfOdU3BkqVEvrazoEioVhPWfaDO10AADwh6QIA4AlJFwAAT8r0nG5o5jynnPW2/dr4/Aufceqm3OeeEBKNT3fVNfHZo/7m1DV5ZJqJ27bZ6j4wsIvNgql2TrcFc7oxS2va2Cnfe/doE59WbZuJw3eT2phnD0M/c4a9hq8c9LLTrlW63XUqbXeJulqokObzrYjI7qZ7kt0FJMpqd+nkEdMuNXGX/Vea+NuvDnbaVV2rpDC76rvfvfn3eWNMfF7Geqeu190TTfyxdDNx5pjEnlAVC/4SAADgCUkXAABPyvTwsmh3+KHV3+1QwuFzb3DqQr02SWE2r8t0ys3es3GlT+3OJo3Dli0EX1nPmOvU/Wf9QSa+/BS7afcPdyT2a/PlTWqbViZ+aMJrTl2bdLvEZ1muHULu9dogp12rZ/4wcZ2VdjlR71fd34+5J42y7U4JmwZ4MrBjTozLEV5441QTN2IJDMqhvE3u39j9zrTl4PEfzWWyxOLVp+wug0+9VM2p++pgu0PgpGtb24q3w3a7KgXLibjTBQDAE5IuAACekHQBAPCkbM/pFqHe82HzBs8X3m7/OLxWat06TrlLNTu3PG1n8zi8QsW04L4MEwfncEVEvthl5+L/9cDNJm72knvdI5320+oKd5vQ8yadbuIJHd5x6o4cYLcQ3X94bPOxjR5kHndfVuftNHGNZaX/nKbqC/mOhk+5q+1JRRmnunW3TT3WxB+3fd/ER157o9PuL3khCbjTBQDAE5IuAACelNvhZZ90Q3eQ+vRq2018y7f2NJws+dlbn8qDl498MWLdo7dcYeI6H5V8yGjRpy1swR2RkmsGjDfxuOF1BYmRmWKnELJr2Lhqgl83tZ1dYnL5tROiflzT0YtNXPoHw+MjtXZtp6z32B3GQjt2+O6O8ek3XUz85MV2KuecG7522n37fBVvfYqEO10AADwh6QIA4AnDy3GwsmediHVp69M99qR8SQ3s+5US9vmw8obs8OYl0uxlO1T4Wh/3cIVjqi408Uf1skyct35DXPtQEWTODnzj9xS3LkPZQyeOusXuBjfnlcT2qeHLdgeyW2sviNiu3Wh3F7MWf06N0LJ8SWvcyMTtP1jp1H34gZ0+azI4sd/QV5Xt78eyQYc6dXf0ej+8eX6fKq0P+0mjQtv5xJ0uAACekHQBAPCEpAsAgCfM6cZBdm2970Yottc2HG3iLg2+c+qW/t3GLR5qb+LQr7/H9Fo6154+siXPPcGkXSX72XTdOXZOt+7I6Jcqbbv4SBOXxoO1fWk8Zqkt3Bq53cHV7Lk0c+SAuPdj8cN2LvLthk8Eaio77UZusfP7rZ5c6NTl5VaMhUJbDm9o4ofrj3Pq7r7mexMfWu/vTl2bUVuL/VqLL6hl4pzaIafu/h7vmvjCDHf+OEWUiYOPeub+8512NSX57z3udAEA8ISkCwCAJwwvo9T67ItDbKGPO7w849gXTLzqA7t86PF13Z12n3zbRaIx9twhJg4/XGF6tv1sut/rv5nYHfwq2vn//MzEE8bUKMYjyxcd2LVo6KZWTt0tte3w7SWZy0z8wCu9nHZtHrMHI4RmzI3qdbdfcIRTnn75kyauGliqFBxOFhEZd56d4sj7M/JyovKs+spdJv7P+oOcun/Wm2Xieec+49SlnBsc8g0u/1NOu2Cd8/go24mIrAsclnHMB7eZOOtd92CT0jARyJ0uAACekHQBAPCEpAsAgCfM6SZAqrKfZWrPTmJHyrhWQxaZ+KeL3O00j6icY+JGafYcmsfDlhY9fpFbjiRF7POHwmZrP9nW0dbt3CmxGDnnGBM3kZkxPUd5kLd5i4m/7O3OD8qHNgzO7y7oPspp9urhdgnRf8e4S0KCLjv3KxvXfNypq6qqhTcXEZGnXjvLKTeak9itDcuEH2eY8Jtbj3KqTv6HnZf/X9u3nLrgtp7h87NB7nIfO+v6+jb39LbzM+x2nR0+HeDUNR1rn6P1Rz+ZuDTM4YbjThcAAE9IugAAeMLwcgLkaTs8WXvO9iJaoih5a9eZ+OFTz3Pq5g3Yz8T9un9p4oF1YtuRqu+yE008dYI77NnihWWB0gqJRZMLKu6QciS5S5c55TeGBo4duiUQ1nZ3groic42Nrx0e5au5w8kvb21g4vfOP8HEjeb8JIgs7ctp7g/sW0/OPOMWp2rVJfaA+ynH2eVE58+72Gm3/kN78o8KzOw0eN1dDja6kx36z/rq56j7XNpwpwsAgCckXQAAPGF4OQGC315GfOTNX+SUWw205a+keiDuGuMr2M3Zm4j7jdWKsa198gUPkPjs5Xom/qJZZ6fd3Bvtt1qPPdxOJ3w3pb1E0nbEJqccmr/ExDpnXvE7i7+oMn6KU24x3sYXi93ZK03caYUDwsp75YWV077aWKL+lRZkBwAAPCHpAgDgCUkXAABPmNNNgEU5dplQ6ma7g1H4HAWAwukcu9wkb8Fip671Lba8NvjzIg4o572H0oI7XQAAPCHpAgDgCcPLcdDsn5Od8oB/HhsouUtdAAAVF3e6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJyRdAAA8UVrrZPcBAIAKgTtdAAA8IekCAOAJSRcAAE9IugAAeELSDVBKaaXUDqXUA1G276uU2l7wuFaJ7h+Kh+tZvnA9y58YrunggvZaKVUmzw7g28sBSiktIq211gsDPxshIieISGsR+ZvW+uVoHofkC78uSqksEXlURI4WkVQRmSoiN2ut5xX1OJQOhVzP40Tkk7Bm1UXkfK31e5Eeh9Ijwt/cziLygoi0E5E5ItJXa/1roL6ZiCwRkXStda7XDscBd7r79puIDBCRX5LdEZRYLREZJyJtRKS+iEwRkQ+S2iPETGv9rdY6Y+9/ItJbRLaLyKdJ7hpipJSqJPnvyddEpLaIjBaRDwp+Xi6QdPdBa/201vpLEdmd7L6gZLTWU7TWL2itN2qtc0TkSRFpo5Sqm+y+IS6uFJF3tdY7kt0RxKyb5B85O0Rrna21HiYiSkROSmqv4oiki4rseBFZo7XekOyOoGSUUtVF5HzJvzNC2dVBRGZod95zRsHPywWSLiokpVQjEXlaRG5Ndl8QF+eKyHoRmZTsjqBEMkRkS9jPtohIZhL6khAkXVQ4Sqn9ROQzEXlGa/1msvuDuLhSRF7RfDO0rNsuIjXCflZDRLYloS8JQdJFhaKUqi35CXec1jqqZQoo3ZRSjSV/LvCVJHcFJTdbRDoqpVTgZx0Lfl4ukHT3QSlVSSlVRfIn89OVUlWUUvy7lUFKqRoiMkFEvtda35Xs/iBurhCRH7TWi5LdEZTYRBHJE5GblVKVlVI3Fvz8q+R1Kb5IHvv2mYjskvy1nSMK4uOT2iPE6hwR6SoiVxdsmrD3vybJ7hhKpI/wBapyQWu9R0TOlvxrullE/iYiZxf8vFwg6bqyRWSaUur+vT/QWnfTWquw/yaKiCilrlZKbS54XCg5XUYRnOuptR5dcP2qB9d3aq2XiXA9y4C/vD9FRLTWbbXWL4Q35nqWCYX9zZ2utT5Ua11Va32I1nr63jql1H2Sv3dCtoiUyfl7dqQCAMAT7nQBAPCEpAsAgCdeT2nomXIBY9lJ8nnoHbXvVsXD9UyeRFxPEa5pMvEeLV8iXU/udAEA8ISkCwCAJyRdAAA8IekCAOAJSRcAAE9IugAAeELSBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgCUkXAABPvJ4yVNYsGH2Iief1GOnUnXTjABNXG/uTtz4BQEWQ2qGNU156bl0TH9ZrllP3StNvTJyj86J6/u439HfKVd+fUtwuxoQ7XQAAPCHpAgDgCcPLRdH2DOKQhJyqld1t3Hqsrw5hr7TmTU28/JyGJt6Wleu0a5O10sTj24wzcdaH1zvtGk2wnz9rTF/j1OntO02c9+efJlZp7ttn1c2Hmzi3qtvfJo9Ns8+XnS0A/mrrpUea+PS7Jjp1Y+vOjPi4HG3fv+F/qyN5dshQpzxoXh8T581ZENVzxII7XQAAPCHpAgDgCcPLMWrZbpWJVeXKTh3Dh/G3ZuDRTvnnQU+ZONrhpGCr+b2fc+t6R36Ot7YdaOIX/36OiVcd5759Zl7pDlcFnTHxWhOr73/dV1eBciulShWnvOhfXUw8+4rhJo72fR2rrPRKTnnOLbVt3fXhreOHO10AADwh6QIA4AlJFwAAT5jTjdHHbd838VkZPZ26POZ04yK1VXMTj77lybDa4v/qjt2+v4nPy1gf9eMuylxt41HPmDgl7DNrcAZqerZbl7pld6HtKpq1N9u5+a2H7S6iZWKlV7ZLy2Yd+1LEdr0bHuqjO+Wfsssvg3O4IiIzrxgWKJX8PrD92zdFrPv9wqci1j104jsmfunw3rZiSuSlSrHgThcAAE9IugAAeMLwMkqtVb3sUp12lSJ/Pjxp5kUmrn5/jYjt0ldvNvELDWo5ddl17fKBAY+849Sdk7Fu350VkVl7tIkH3TbAqas2i0MxRER2HGl395pzwsiI7YJD97EuHYn2OYI1r21tHNNr4a9Cx9lh5MX97M9/P2lYIa3/6t3tBzjlf35nl+s1Huf+Paj6gT2soJX8aGLVpYP7pBdGfr3g+3xYi+omzozzOQjc6QIA4AlJFwAAT0i6AAB4wpwuSq1jr5gWsW513i4Tr51Z38Spp0V+vvo/23nbtYeluq/Vwy4LiHYON9yHWzubuNpY5nAL03rAEhOfm3mOU7fkqiYmzq5tZ1qVlpiE6u0x8Zwez0ds1/ZjO//e7o6FYbWbYnvxiiiwLEgkfB53RFRPcca8M00cunc/py7r+59j71spwp0uAACekHQBAPCE4WWUWh/93MnEj5zxrVPXJC3DxHMuHS5RudqG6codXs7ReYGS+1l0fWAo+7h3bjfxxAsec9rdXc8OUXe78AanLuPtHwUieZu32EIwFpHG96+I62ttv9AeiC493LqFOXZHqnaPbrT928RwcnEETwwK32kq2qVBP2Wnm1iftNLESlYW1rzM404XAABPSLoAAHjC8DJKraz+diuYQ+r2depmHvOyiWPZsSgn7Bux43bYA6yHLunu1KUMrWfilh/bYeLjqt/qtJt7xtMmXtUzz6nLervYXUQJre69J2Ld4BV2Q/u8+Yt8dKdc0u1amtg9uCCydl9e55RbjrDv3xT5NT4dK8W40wUAwBOSLgAAnpB0AQDwhDldlAktBrvzc9063BChZWxq/bzGxFUXLwmrDS/v28FZy51ydiydQoks6D7KxKGw+4tpU1qbuJVs8Nan8mZl95omTiniHm7sjjombj08x62M8yHxRQn28a/LBm2s3c214twHAADgBUkXAABPGF5GmZA3e55Tzpgd3+fP3XeTv2iTFXnHnJnz3cPQs2RNhJZIlJDoQOwuK4v1EIWKLq1xI6d86qWTTVzU0r07v7rIxFlT4nwqfBFW3OuWg30MXzZ45VK7bVntj343sbv4r+S40wUAwBOSLgAAnjC8XJTAGFT4N/PCv/mGiiGnx6EmntDGPSN0cmDj9jbP7HTqGM1MvF1nHR72k8jnMefVsd+gXfyGPQf50KbLnHYDD/zcPkbcr7Re++KNJm78nx+K09Uya+Nx7vDyf+qPjdi256wLTdzujrkmjvdwbbilb3U08YudX476cYuea2viWlsnF9GyZLjTBQDAE5IuAACekHQBAPCEOd2iBLYlCf86fPDr5nMebOnUZV23UVB+pGRmmvjBEXYeN3xe/5vtdk5IT4/zmqZyKLX+/k5529HNTbyrjr0fSDl3fVTPN7rDkLCfVI7Ydu7Jz0X1nH3/6GniaZ+2d+qaPWFPxCn+OVdl04Yzd+67UYHlK+qaOGtr8Xd1i9UdHT8z8WGVI88g9112olOu++lCEydy3pk7XQAAPCHpAgDgCcPL8VCpogwuVQypdes45e1v2E3du1SOvKPNi5NOMHFr+SkxnSvjck4+zMSZ9y516sa2GG7i4BK9onY6cqXvu0mB4LDxn7c2idzwxxkmbCLusqCK+K6/u/OnTrmoQw6y+v6c6O4YWz+xU3x9agSXikXu3+8vdnDKdf9M3DKhIO50AQDwhKQLAIAnJF0AADxhThcIs7xvW6f880FDC233n/UdnXK7J9eaOJZTiyqCP06zf3ImtJjg1L2+raGJN+dVM/EHqzo57dZ93VAKM6zv8065e1W78KPrL5c4dXV6zw+UNhfdaRh52r1Pi36+veRSa9nvVix8rqlTN7vjS1H1qf3bN5m41Ug/c7jhuNMFAMATki4AAJ4wvIwKKbjLlIjI2tcbmPi9To+Gta5komGb7NDzhEeOc1rVXPxj/DpYTtWaY3d5y/r4eqeu3SA75Ju3eYuJK8kfTrtGYeW9frvUHXI8vsqCmPuJ5Aue6CUiUv9+ez3HNnkhrHXh949f7HLf521G2t0CE33aUSTc6QIA4AlJFwAATxheRqm1ZuDRJq7Uw930/vH2b5s4pKP77PjA0tNNPLj5+05dcKep4HByuK8vsjwmBRwAAAQASURBVDsq1ZzNcHJx1RsxORC7dYkc7kt/rc6+GyGuNlxzlInrjorum8LzX7JDyk0bbnDqRjb5sth9uOmTK51y69+Tv1Mcd7oAAHhC0gUAwBOSLgAAnjCni1Jj20VHOuWfBz0VsW3wAPkcnRPV83/c1s7jhh9AHzwxaEtot1PX/bFBJj5gtnvSDJIrtf7+Jm6QXvhSIhGRtN0V8Uyg+HtkxslOuc+xL0VoKdK+72wT/3yg/X5Gv4s/dtrdUGuRidPVrybO0eGz/JHvEYPv56zRN5q49T+Ss+tUUbjTBQDAE5IuAACeMLwcRqXZf5LK1fcksScVz+qe7jEBRW1cHhwOjmXT9fAD6IPP8X9rujt1DT/708TJ2sUGhdt2dHMTn5PxUVgt9xTx1mhEulOe3NUO6x5R2Z3mcZb4XB95uU/w3Rvt+zq4M5yIyMjxdti7xb9+MXHY27xU4LcSAABPSLoAAHhC0gUAwBPmdMOkNG9i4l+PfjFiu+CykgYf888Yq9S6dnu+Sw6dksSeWE82+NYpfz0+w8RPHdvNxLlr1gpKj5Swe4jgezRtO7Px8ZD25TSnfPvg/ib+9sFhcX2tFbnZTvmRtT1NvPyqxk5d89/t0qDSOI8bxJ0uAACekHQBAPCEcdFwGzeb8OBXbjbxYcfPdZqteLS1iTPeT/7JFWVVTnt78Ph9+0+I+/Of+vv5Jl47qaGtUG67uy6zpxZdlLnaqTux6nYTP1U58glESK7wJSavbDnYxOlfTAtvjjio96ndTapL41ucuun9h5bouc8ZeodTPvCJ4G5w80v03MnEnS4AAJ6QdAEA8ITh5TB5GzaauHlgs+wNYe2qSun4pm1Zl77aDucfO/0yp+67Lq9HfNzqvF0m7vmqPZCg1YgVTrvKq+xQceOcyBvij3mui4nfqnaUU7f10AYmztxadoe1KpqRc44xcROZmcSelF95a9eZuPF/1jl1Z/6na4me+0Apn4eLcKcLAIAnJF0AADwh6QIA4AlzukiqvIVLTFynt1t3pkQ3J9RM7Nx7bhHtiuzHn39GrKv2x3LbLsbnR2Ks7BG5LuPjjMiVQJJwpwsAgCckXQAAPGF4GUCZlbLbbi32+IaDnLo6L00Obw4kHXe6AAB4QtIFAMATki4AAJ4wpwugzGp5248mniRVk9gTIDrc6QIA4AlJFwAAT5TWOtl9AACgQuBOFwAAT0i6AAB4QtIFAMATki4AAJ6QdAEA8ISkCwCAJyRdAAA8IekCAOAJSRcAAE9IugAAeELSBQDAE5IuAACekHQBAPCEpAsAgCckXQAAPCHpAgDgCUkXAABPSLoAAHhC0gUAwBOSLgAAnpB0AQDwhKQLAIAnJF0AADz5f3XpyC6H9WCIAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 定义画图模板\n",
    "plt.figure(figsize=(8,8))\n",
    "# 随机输出16个图片\n",
    "for idx in range(16):\n",
    "    plt.subplot(4, 4, idx+1)\n",
    "    plt.axis('off')\n",
    "    plt.title('[{}]'.format(mnist.train.labels[idx]))\n",
    "    plt.imshow(mnist.train.images[idx].reshape((28,28)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，定义用于训练的网络，首先定义网络的输入。\n",
    "\n",
    "这里我们直接使用上面的数据作为输入，所以定义两个placeholder分别用于图像和lable数据，另外，定义一个float类型的变量用于设置学习率。\n",
    "\n",
    "为了让网络更高效的运行，多个数据会被组织成一个batch送入网络，两个placeholder的第一个维度就是batchsize，因为我们这里还没有确定batchsize，所以第一个维度留空。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 定义模型（计算图）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n"
     ]
    }
   ],
   "source": [
    "# 定义输入x和输出y变量（占位符，未赋值）\n",
    "x = tf.placeholder(\"float\", [None, 784])\n",
    "y = tf.placeholder(\"int64\", [None])\n",
    "\n",
    "# 定义学习率（占位符，未赋值）\n",
    "learning_rate = tf.placeholder(\"float\")\n",
    "\n",
    "\n",
    "#运行了几轮BATCH_SIZE的计数器，初始值为0，设为不被训练\n",
    "global_step = tf.Variable(0, trainable=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 优化参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 学习率参数\n",
    "LEARNING_RATE_BASE = 0.9 #最初学习率\n",
    "LEARNING_RATE_DECAY = 0.99 #学习率衰减率\n",
    "LEARNING_RATE_STEP = 128 #喂入多少轮BATCH_SIZE后，更新一此学习率，一般为：总样本数/BATCH_SIZE\n",
    "\n",
    "# 网络结构\n",
    "L0_units_count = 784 \n",
    "L1_units_count = 280\n",
    "L2_units_count = 10 \n",
    "\n",
    "# 正则参数lambda\n",
    "lambda_ = 0.0001\n",
    "\n",
    "# 一个批次取的样本数量\n",
    "batch_size = 128\n",
    "\n",
    "# 训练次数\n",
    "trainig_step = 20000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### BP计算"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "隐层节点数计算3个公式：\n",
    "$$ \n",
    "m = \\sqrt{n + l} + \\alpha \\\\\n",
    "m = log_2 n \\\\\n",
    "m = \\sqrt{nl} \n",
    "$$\n",
    "\n",
    "m: 隐层节点数；\n",
    "n: 输入层节点数；\n",
    "l: 输出层节点数；\n",
    "$\\alpha$: 1-10之间的常数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 从截断的正态分布中输出随机值，stddev是标准差\n",
    "def initialize(shape, stddev=0.1):\n",
    "  return tf.truncated_normal(shape, stddev=0.1)\n",
    "\n",
    "# 隐层神经元的权重\n",
    "W_1 = tf.Variable(initialize([L0_units_count, L1_units_count]))\n",
    "# 隐层神经元的偏置，几个神经元就有几个偏置\n",
    "b_1 = tf.Variable(initialize([L1_units_count]))\n",
    "# 计算隐层神经元未经过激活的logits_1\n",
    "logits_1 = tf.matmul(x, W_1) + b_1\n",
    "# 对logits采用relu激活函数\n",
    "output_1 = tf.nn.relu(logits_1)\n",
    "\n",
    "# 输出神经元的权重，输出神经元每个神经元的输入数量=前一层的输出数量\n",
    "W_2 = tf.Variable(initialize([L1_units_count, L2_units_count]))\n",
    "# 输出神经元的偏置\n",
    "b_2 = tf.Variable(initialize([L2_units_count]))\n",
    "# 计算输出神经元未经激活的logits_2=前一层的output_h1 * W_2 + b_2\n",
    "logits_2 = tf.matmul(output_1, W_2) + b_2  \n",
    "# 把第二层未经激活函数输出的logits_2赋值给logits\n",
    "logits = logits_2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 交叉商损失+正则、优化算法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来定义loss和用于优化网络的优化器。loss计算使用了sparse_softmax_cross_entropy_with_logits, 这样做的好处是labels可以不用手动做one_hot省了一些麻烦。这里使用了sgd优化器，学习率为可以根据需要设定。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 设置正则化方法\n",
    "regularizer = tf.contrib.layers.l2_regularizer(lambda_)  # 定义L2正则化损失函数\n",
    "regularization = regularizer(W_1) + regularizer(W_2)  # 计算模型的正则化损失\n",
    "\n",
    "# 使用softmax交叉商损失，sparse_softmax_cross_entropy_with_logits可以不用对labels进行one-hot\n",
    "# tf.reduce_mean()对batch_size里每个样本的loss求平均\n",
    "cross_entropy_loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=y))\n",
    "\n",
    "# 总损失等于交叉熵损失和正则化损失的和\n",
    "cross_entropy_loss = cross_entropy_loss + regularization\n",
    "\n",
    "# sgd优化器\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).minimize(cross_entropy_loss)\n",
    "\n",
    "#定义指数下降学习率\n",
    "learning_rate_decay = tf.train.exponential_decay(LEARNING_RATE_BASE, global_step, LEARNING_RATE_STEP, LEARNING_RATE_DECAY, staircase=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 评估模型性能"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "需要注意的是，上面的网络，最后输出的是未经softmax的原始logits，而不是概率分布，要想看到概率分布，还需要做一下softmax。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将输出的结果与正确结果进行对比，即可得到我们的网络输出结果的准确率。\n",
    "# 对logits做softmax得到输出概率分布\n",
    "pred = tf.nn.softmax(logits)\n",
    "\n",
    "# 评估模型性能：对比模型的输出和labels有多少能匹配的上就行，如果y==pred则True，否则False\n",
    "# tf.argmax(input, axis=None, name=None, dimension=None)： \n",
    "# 此函数是对矩阵按行或列计算最大值，axis：0表示按列，1表示按行\n",
    "# 返回：Tensor  一般是行或列的最大值下标向量\n",
    "correct_pred = tf.equal(tf.argmax(pred, 1), y)\n",
    "\n",
    "# 获取准确率：因为是按batch进行模型训练，所以要对batch做均值处理。\n",
    "# 因为correct_prediction是Boolean类型，所以把它强制转换成float32类型\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))\n",
    "\n",
    "# saver用于保存或恢复训练的模型。\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 训练模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上定义的所有操作，均为计算图，也就是仅仅是定义了网络的结构，实际需要运行的话，还需要创建一个session，并将数据填入网络中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 1000 training steps, learning rate is 0.838859, the loss is 0.152281, the validation accuracy is 0.9726\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-1000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 2000 training steps, learning rate is 0.774053, the loss is 0.0971961, the validation accuracy is 0.9778\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-2000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 3000 training steps, learning rate is 0.714253, the loss is 0.0948199, the validation accuracy is 0.9804\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-3000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 4000 training steps, learning rate is 0.659073, the loss is 0.091788, the validation accuracy is 0.9804\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-4000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 5000 training steps, learning rate is 0.608156, the loss is 0.0735727, the validation accuracy is 0.9812\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-5000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 6000 training steps, learning rate is 0.566842, the loss is 0.08042, the validation accuracy is 0.9826\n",
      "WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/training/saver.py:966: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to delete files with this prefix.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-6000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 7000 training steps, learning rate is 0.523050, the loss is 0.0646858, the validation accuracy is 0.9826\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-7000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 8000 training steps, learning rate is 0.482642, the loss is 0.0605246, the validation accuracy is 0.9842\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-8000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 9000 training steps, learning rate is 0.445355, the loss is 0.0626286, the validation accuracy is 0.9846\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-9000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 10000 training steps, learning rate is 0.410949, the loss is 0.0582584, the validation accuracy is 0.984\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-10000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 11000 training steps, learning rate is 0.383031, the loss is 0.0557529, the validation accuracy is 0.9834\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-11000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 12000 training steps, learning rate is 0.353440, the loss is 0.0565671, the validation accuracy is 0.984\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-12000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 13000 training steps, learning rate is 0.326135, the loss is 0.0547404, the validation accuracy is 0.9836\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-13000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 14000 training steps, learning rate is 0.300939, the loss is 0.0532613, the validation accuracy is 0.9838\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-14000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 15000 training steps, learning rate is 0.277690, the loss is 0.0499291, the validation accuracy is 0.9842\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-15000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 16000 training steps, learning rate is 0.256237, the loss is 0.048686, the validation accuracy is 0.9846\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-16000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 17000 training steps, learning rate is 0.238830, the loss is 0.0542577, the validation accuracy is 0.9846\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-17000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 18000 training steps, learning rate is 0.220379, the loss is 0.0443857, the validation accuracy is 0.9846\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-18000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 19000 training steps, learning rate is 0.203354, the loss is 0.0469855, the validation accuracy is 0.984\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'./model.ckpt-19000'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the training is finish!\n",
      "the test accuarcy is: 0.9837\n"
     ]
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    # 初始化模型里的变量\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    # 定义验证集与测试集的变量值\n",
    "    validate_data = {\n",
    "        x: mnist.validation.images,\n",
    "        y: mnist.validation.labels,\n",
    "    }\n",
    "    test_data = {\n",
    "        x: mnist.test.images, \n",
    "        y: mnist.test.labels\n",
    "    }\n",
    "    for i in range(trainig_step):\n",
    "        # 从mnist的训练集里获取一个批次进行训练\n",
    "        xs, ys = mnist.train.next_batch(batch_size)\n",
    "        learning_rate_val = sess.run(learning_rate_decay, feed_dict={global_step: i})\n",
    "        # 运行optimizer和cross_entropy_loss，并且给模型的placeholder图进行赋值\n",
    "        # 返回值：_ 表示的是optimizer的返回值，但是我们不关心这个返回值，所以用下划线接收\n",
    "        #     loss 表示的是cross_entropy_loss的返回值\n",
    "        _, loss = sess.run(\n",
    "            [optimizer, cross_entropy_loss],\n",
    "            feed_dict={\n",
    "                x: xs,\n",
    "                y: ys,\n",
    "                learning_rate: learning_rate_val\n",
    "            })\n",
    "\n",
    "        # 每100次训练打印一次损失值与验证准确率\n",
    "        if i > 0 and i % 1000 == 0:\n",
    "            # 在验证集上的准确率，调用run方法前，需要对之前的placeholder变量进行赋值\n",
    "            validate_accuracy = sess.run(accuracy, feed_dict=validate_data)\n",
    "            print(\"after %d training steps, learning rate is %f, the loss is %g, the validation accuracy is %g\"\n",
    "                % (i, learning_rate_val, loss, validate_accuracy))\n",
    "            saver.save(sess, './model.ckpt', global_step=i)\n",
    "\n",
    "    print(\"the training is finish!\")\n",
    "    # 最终的测试准确率\n",
    "    acc = sess.run(accuracy, feed_dict=test_data)\n",
    "    print(\"the test accuarcy is:\", acc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 测试模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用我们训练的模型做一个测试。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to check for files with this prefix.\n",
      "INFO:tensorflow:Restoring parameters from ./model.ckpt-19000\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 576x576 with 0 Axes>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.0\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x133a039e8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '7: [7]-[100.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1339fce10>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x12b8477b8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '2: [2]-[100.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x12b86c6a0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x108d70198>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '1: [1]-[99.9%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x108d7cef0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x108d938d0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '0: [0]-[100.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1283aefd0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1283be278>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '4: [4]-[99.9%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1283df668>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1283df9b0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '1: [1]-[100.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x128415240>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x128415128>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '4: [4]-[99.9%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x128438710>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x128438a58>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '9: [9]-[97.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x12846f240>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x12846f278>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '5: [5]-[79.4%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x12847f080>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x12848f6d8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '9: [9]-[99.8%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1284a87f0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1284c6208>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '0: [0]-[100.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1284e95c0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1284e9ac8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '6: [6]-[100.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x1285028d0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x12851f438>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '9: [9]-[100.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x12852bfd0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1285428d0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '0: [0]-[100.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x12857a390>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x12857a278>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '1: [1]-[100.0%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x12858a240>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x128d3c9e8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "(0.0, 1.0, 0.0, 1.0)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "Text(0.5, 1.0, '5: [5]-[99.8%]')"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x128d6e0b8>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAd0AAAHiCAYAAACtERYWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzs3XeYFEUaBvD328ASlpwlLTmDCmLAAIpiFgMmRDAeImI48QyomMOdCTMqomLgBLMnBhQFkSygRBUQyXHJLLD73R/d2901Tu/OTujZ8P6eh4evp2qqq7dDTVd1EFUFERERJV5KsitARERUVrDRJSIiCggbXSIiooCw0SUiIgoIG10iIqKAsNElIiIKCBtdIiKigJS4RldERojIARHZJSKVIvzOHyKyX0TGFpBHRWS3iDwUv9oGT0S+FZF9IjI12XWJFNdpwUriOvUq6+tXRDLsZT8gIg8muz7REJEx9vpYGWH+VvYy54rI1T55eohInp3v1LhWOEBFXb9JaXTtCnr/5YrIs0UoYpyqZqrqbru8L0LK2y8iv+RnVtXmAB6OoNzOqnqXp56jRGSpvWEMDLMcN4vIehHZISKjRSTDk5YlIt+JyB4RWSIivfxmaq+00XY560XkFk9aIxGZLiJbReSJkO99ISJdvZ+p6okABkWwrHFj1/81EflTRHaKyDwROa2IxYSu02Ei8qtd3goRGebNzHUaLBEZIiKzRSRHRMZEUUTo+u1p/y23hzuQx7B+z7K3m10iMk1E2nnSMkTkKRFZKyLbROQFEUn3K7iQsk6yt8v1InKx5/NqIjJXRCp7liVHVTMBvB3B8iSMiNQQkQ/F+qHyp4hcWsQiHlfVLE95vtu4qi6zl3lKIWWutbeLiXaZ9UXkE3sdqYhkeTMXNE87/SR739xjb19N/GZc0P6cyPWblEbX/iNn2hWtB2AvgPdjKO+0kDKnxVKex3wAgwHMDU0Qkd4AbgdwEoAmAJoBuM+T5V0APwOoCeAuAONFpLbPfEYAaGmX0xPAbeL+8rsDwBsAmgLok39AFpGLAKxQ1dkxLF+8pAH4C8AJAKoCGA7gv6E7TBEJgMsBVAdwKoAh3o0/Blyn0VkL4EEAo+NU3m67rGGFZYyUiLSEdeAbBKAagE8BfCIiaXaW2wF0BdABQCsAh8PaVqMp62kAZwHoDeAFEUm1P38EwKOqujNeyxVHzwPYD6AugH4AXhSR9jGUNwL+23i08gBMBHB+UecpIrUAfADgbgA1AMwGMK6AeRW0Pydu/apqUv8BGABgOQCJMP8IAGMLSM8CkAsgq4jfUwAtfNKmAhgY8tk7AB72TJ8EYL0dtwKQA6CyJ30KgEE+5a8FcIpn+gEA79nxFwBa2/F7AC4EUMXeWKr5lDcQwNQkr9cFAM6Pxzq184wE8CzXafLWqV2PBwGMKeJ3fNcTgF4AVkazXYSuXwBDAHzumU6B9YP+JHt6NoC+nvRLAfzlU3ZhZS33pK0HUAdANwATC6jvGAAPJmm9VYLV4LbyfPYWrAYkku//re4FbeOezyYDuNqnzB4AVvukpdnrNyvSeQK4FsC0kGXeC6BNmPIL3J8TuX6Lw5juAABvql1rABCRbBE5NsryLgcwRVVXxqNyBWgP66wp33wAdUWkpp22XM1fQ/Ptzw0iUh1A/TBl5ef9FcDJIlINQBcAC2FtaE+ranacliWuRKQurI16oeezqNepiAiA47zlJQjXaZRi3GfjSUJigXVm65feUESqRlHWRhHpLCKdYZ2dbQPwDIChMdQ9kVoBOKiqyzyfOdukiDS212HjSAqLYBuPuwjmaey/ag1l/OFTp8L254St36Q2unZ/+wmwutocqlpNVaO9aORyWL84Ei0TwHbPdH5cOUxafnpl/F1myPdD8z4Cq8H5HsALAMoB6ATgUxF5R0R+EJEh0S5EvNljZG8DeENVl+R/HuM6HQFrW3099hoWiOs0SjGu33j5BsAJYl2gUw7AnbD+thXt9IkAbhSR2iJSD+4BtOLfiyq0rEGwDsKjAPQHcJ39nfIi8qU9VnhC/BcxapkAdoR85myTqrrKXoerilBefhl/Ky9BCptnUffRgvImbP2mFZ4lofrD6jJbEY/C7F/a9QCMLyTfF7AOegDwD1WN5gKHXbC6BPPlxzvDpOWnhxsH2OVJ3xeaV1W3ArjIrncKgB9gbRC3wzpjGghgrohMUtXFUSxH3Nj1ewtWN1ZcGg278bkcwHGqmlNAPq7TUiyS9auqS0RkAIDnYJ0RjQWwCMBqO8tDsMZn58HqWnwFwGEANhS1LFWdB6t7FCJSH8ATAI6G9UPqJljdoD+ISBNvL14SFWX7jbS8/DL+to2HIyK7PJPt/PLFMM+i7qO+eRO5fpPdvXw5Qs5yYzQAwAeququgTGpeeBXtFYULAXT2THcGsEFVt9hpzbxXuNnpf+seVdVtANaFKStcV+q1AKar6q8AOgKYrar7AfxiTyeN3QX8GqyLNM5X1QNxKPNK2Bc2qerqgvJynZZuka5fVR2vqh1UtSaAe2Fd4zHLTturqkNUtYGqNgOwBcAcVc0ralkhngIwXFX3wl2HKwGkA/C70C5oywCk2ReI5fPbJgtVxG08/zuZnn+RnlEXZZ7G/ivW7WnNfeoU8f6MOK/fpDW6InIMgAaIz1XGEJEKsC5IGROP8uwyy4lIeVhjOekiUt4+MwGANwFcJSLt7LG54fnztsdN5gG41/7OubC6Dyf4zOpNAMNFpLqItAFwTehyiEgdANfD6moFgBUAeopIJqwrMpfHYZFj8SKAtgDOsjfOmIhIP1i3jJysqnFbNq7T6IhImv13SwWQav8Nou4pE5EUu7x0a1LK2924sdazi4ik2lehjgLwSf4wh4g0EJFDxHIUrKtc742mLE+ekwGUV9XP7I9WADjRvio4A1bDnnT2+OYHAO4XkUoi0h3AObB6pqJV6DYeDXu7yL9VL8OejmSeHwLoICLn29+5B8CC0HUGRL4/J2T9FnalVaL+AXgZwFs+abtgdSeGSxuBMFc0ArgEwJ/wuQra73ue9L9d6QrryjsN+dfDk34LrK6pHbDGGzM8aVn29/cCWAqglyetH4CFnukMWLdP7LDLuyVM/d6EeeVlIwAzYA3wPxmSdyACvNIV1uX7CqvLZ5fnX79o1ymsjftASHkvcZ0mbX8dEebvNiKG9dsjTHmT47B+p8LqItwK6xhTyZN2PICVAPbY669fyHe/AHBnJGV51vE8AE08n51kz2MdgItD8o9Bkq5etudfA8BHsG7XWgXgUk9aY3sdNvb57t/qHuE2PhlFvHo5zHahkc4T1tXwS2Dto5PhufoZwEvwHENQwP6cyPWblJUf44Yz3N5oskN3ggK+s9TeoEYXkGcfrIH0B5K9jDH+fb62DxSTkl0XrtOyu065fo16ZtjLvhvAvcmuT5TL8Iq9Pv6IMH9Le5n3IOTWPE+e4+0GLxtA72QvY1DrV+wvERERUYIl+0IqIiKiMoONLhERUUDY6BIREQUk0IdjnJzSlwPISfJ13vtSeK6i4fpMnkSsT4DrNJm4j5YufuuTZ7pEREQBYaNLREQUEDa6REREAWGjS0REFBA2ukRERAFho0tERBQQNrpEREQBYaNLREQUkEAfjkGUCCsfPNqJc8ubzwKo3X6TE//U2e/Vt0Dzb69w4sozKxhpdUdOi7WKREQAeKZLREQUGDa6REREAWGjS0REFBCO6VKJtO3zlk7866HPRfSdAwU8+n1Jz1ed+O2u9Y20/359ghPnLv4twhpScSJd2hvTn3/ylhN3fGmIEzd6gOP3QUutVtWJlz7XzIm9+yQADN/YxYl/6dfKSMtdtCxBtYs/nukSEREFhI0uERFRQNi9TCWCtzsZAH489L2IvvdStttd9eRPJztxVpNNRr6v2n3gxP0qrzPSHhpYy4mb/YvdyyXRxiOqGNMHkevEFdfylbPJlNe0oRP/0uNlJw4dDnqwzhwn7nzuMUZaI3YvExERUSg2ukRERAFh9zIVWwdPcq9W/Lbz8yGp6U709Db3SsbvLupqZlu70QlbbZvtxCnlyxvZHp7R0YnvrPWLWY/qByOuMxVP2zrlGtOrD+Y4cc3Xfgq6OmVaWqOGxnTTUb8nqSbJwTNdIiKigLDRJSIiCggbXSIiooCU6DHdLdccbUw37u+ODSzZWNdI25/jjgE2eNeNK67eZeTLm7conlWkGOxqUM6JU0J+H3rHcSef7Y7H5i5fGlHZv993mDH9To0nPFMZRlrDifxtWhJp90OdeMqZTxppJ/xwgxO3wM+B1amsWnWPe4tPl1PNY+zj9acUubzMY8xb/v662y2/1gL3GowKH88sctmJxqMJERFRQNjoEhERBaREdy/fNuwdY/r8StvcieYFfLGHG648uMdIemZTz9grFqGZG5s4caUnqhppaZPmhGYvc6q96d7KccHsy4w02bbDiQ+uW1nksq8+/RtjOjMlwycnlVRb21Vw4vqpFY20BuPTQ7NTAi34x7NOfEBzC8gZmcmd3zY/6OyGH+52X1gyemcfI1vat8k/rvJMl4iIKCBsdImIiALCRpeIiCggJXpMd+SdFxvT93Ryf0NUX2y+omJbW3Hicp2ynfjxDh8Y+Z6qP8OJP9+T6cRnVDRvLSrIXt3vxDNyKjlxj/IHzIyeebW46B9GUqtJEc+uTIjHS6pXPuTeYnZVtf+EpLqPhfznuqOMlMrfLHbrEXMtKCgnDXavCfhodzUjLXOye2sZ12lipE92x1bTJTXm8n7en+fEKw/UNtLOrbTViS/MdB/9euFbo4x8ZzbogmTjmS4REVFA2OgSEREFpER3L1caPyNk2j9vFZ/Pn63Xw5h+sHuW+53v3SdcPd6jRcT1StvrdoNUWuC+EL3mDxOMfB3LeZ6MtZK3MCRCdn+3S/nHy90u5aop5luGfspxu7/mPWg+rarCjuL3VBv6u9T2rY3ph+u868Sv7TDfbJObvT2QOpUle/t0M6avqP++E3tvE4r0lqEOkwYZ07Unubf1ZWw3y7ijh3v++Evfkb5lrr7DfXJVw0emRVSPeOOZLhERUUDY6BIREQWkRHcvx8PB9RuM6UoT3GlvB0al8VuiKn/D1W73Zvty5p/7P1vd7rCs15eb9YpqbhRq8+HuVeyhXcpeAyZf7cStPmJ3ckm05uSavmlzdjYJ+WRvYitTRni79B980rxSuGu5/d6cvmV4nyA1/LvznbjtbUuMfLk7dsBP69/cF6DMPNvdz7tl7DPyfXHd4058SvnbjLSsh92nVWlOju+8YsUzXSIiooCw0SUiIgoIG10iIqKAlPkx3URIa9LIiZ+78zknDn0qy/vP9HLimut+AsVu/9fm2N1Pbbwvp3fHejr/NMDI1/affzgxn1BUMu1od8A3bd5zhxrT1cD9LR7yPNepmGO4/q7881RjeudF7tugWq12r6coyn7ofWLd4DHurUaz//G0ka9+qjuvuVeZaed/4B4TdP5iJArPdImIiALCRpeIiCgg7F5OgCU3N3DiIzLcFy0s3G/eplBj0Z7A6lSapTXLcuIHWrxvpFX33CY0x3MXQJMHzM6r3G3bElI3Sqyc045w4o9PedZIu3+z+3D7GhMWGGl5oCDduaGrE++42ry1K3f1b3GdV9aEzU58dx/z5SWP1psV13lFg2e6REREAWGjS0REFBB2L8dBzhlHGNNzL3jKM+U+pPu6G2808lWYxicfxUPz/65x4sPK+f+OvMTzAPVW85PfzUSxW32iewjrVM584tiAlR2duM5u8+lGFH8FvTN3weHe95vHtzv5b8Qd0ktLMQcSCqrj2vvcuF6fuNfKwTNdIiKigLDRJSIiCggbXSIiooBwTDcOVp1m/nbJFHcc95IVJztxxYnzjXwKita2Ae7bm+6r633qVIaRb8BK96lfbW/73Yn51KnSoXaHjU6cq+b4XdrH1YOuTpmz9LqKThzpy+kTbeV57i1J42ub180c0FRPbNb3kHvdOJG3lPFMl4iIKCBsdImIiALC7uUopVSu7MT9j5tqpO3Ic1+cvPHhZk6ckcPbVKKV1uAQY/q4oTOcODMlIzS746dFLZy41Tb+/UuDtKbuSy3+09p9Atkr2xsZ+WqM5ksNEm34cZ8mZb5pjRoa0zu7uMeHl654IaIyZuaYt5jJ/oOxVywCPNMlIiIKCBtdIiKigLDRJSIiCgjHdKP024j2TvxZLXMM4ZzfznfijP9xHDEeFt9pjtd9VC/8WFLPX/oa07xNqPT57R/u+N1RnuH8a+b2NPI1wq9BVYkCtui+esb0wlOei+h7E3bVcuIXbzWPFeUXB/NYXp7pEhERBYSNLhERUUDYvRyh7ZeZL0NecNFIJ/7j4AEjbddj7uXsGViX2IqVEXPOfirkk/C3CVUdbD5L5iBfTl/q5DXaF/bzvdnlw35OpUP65PpO/Ej9CVGVMWbNMU5c/tPkvOWNZ7pEREQBYaNLREQUEHYvF8D7FKSb7h5npGWI+6e7eH5/I632F7xiOVkO1K1qTKfvb1DkMnI3bTamNSfHiSXD7dZOrV0LfnJrVzOmf/tnuYjmrbnuC7jb3PC7kZa7Y0dEZZR2Lxw5NuznDb7wf0E5JUaquMM5Bb0gfselR/mm3Xf/a07cs0L4oYPQ8v/+coXI1r2euCaifInEM10iIqKAsNElIiIKCBtdIiKigHBMN4SkuX+Szp+tduK+mVuMfG/vrOPEde82f7sk8gXIVLDPx4+OuYxjfr7EmN68oYoTV6+904lndHkn5nkVpN3wIcZ0s9vK5ltz9p3VzZg+trz3Vg8ewpLp0XEXOPGFVz3tm++Hfz/vxAW97P6ARjbfgsrw6jBpkDHdEnMjm0EC8UyXiIgoIGx0iYiIAsK+mVCdWzvhA3Xe8s32/MPuw7KrzS+b3X5BOmdRP2N6UofxCZvXtMPejep7e3S/Ex9Q/0GG0xcMdOLt8/xvO2owNZiXahd3q842+xy9t+vdv7mjE2d+PMfIF2FPJcWg2Tj39rqZl5lPBOuW4X/7T6xCX0A/av0JTrxtsPsyhDYrQm67S1iNIsczXSIiooCw0SUiIgoIG10iIqKAlPkx3dR2rYzpa9/7OGy+dqOvN6az3pqesDrR31XovcKYbv+wezuNRrgVV26z1YmLcrtP+ylXuPNaVck3X7Pxu9yJmb/45quO38LG5Eqt4t6m9a/u//PN984Xxztxs4O8tiJouYuWOfE9t1xtpP11lntdw7LTXo7rfAePNm8FavTQNM9U8X6zGM90iYiIAsJGl4iIKCBlvnt5yeDqxvRZFcO/yaXh5P3mB8obEpKp6Z2xdSWeiS6RzwsLYpoXFV2e581Oi/YcYqT1WtPViVs+vNCJi8PtIGVZhY/Nl8K38ozUHX+JOzyXPnCDkW9ie/cNbqf8erET542pY+RT9wVcyJq3yUgrSeueZ7pEREQBYaNLREQUkDLZvex9gPqks54ISa0YbGWI6G/U0728tKuZVg5/OnFJ6lYsy6q867nbI+SBb+fCPR5XwnJPynL4KcnrnWe6REREAWGjS0REFBA2ukRERAEpk2O6a7unOnHjNP8xXO+L6tN3mLcM8YYhIiIqKp7pEhERBYSNLhERUUDKZPdyQR7Z0s6Jf+qd5cS6zv8B9kRERJHgmS4REVFA2OgSEREFhI0uERFRQMrkmG6z29031Jx+++EF5Fyf+MoQEVGZwTNdIiKigLDRJSIiCogoX8ZOREQUCJ7pEhERBYSNLhERUUDY6BIREQWEjS4REVFASnyjKyJjRGS/iKyMMH8rEdklIrkicrVPnh4ikmfnOzWuFY4zEell1zNPRHoluz7REJERInLAXo5KEX7nD3u9jy0gj4rIbhF5KH61DZ6IfCsi+0RkarLrEg3uoyV/H/Uq6/trJNtnQYpVoysiLe2Di++K8fG4qmaFKa+GiGzyHqxUdZmqZgKYUkiZa1U1U1Un2mWJiNwlIqtEZIeIvCciVTzzaiAiH4vIVhFZLSKDCljOwsoaJiKbRWShiHT0fN5dRD7ylqWq39jLs6qQ5UkoERkiIrNFJEdExkRRxDj7773bLq+niHwnItvDHaxVtTmAhyMot7Oq3uWp5ygRWWofAAeGWY6bRWS9vV5Gi0iGJy3LrtMeEVlS0AFURDLs7++wy7vFk9ZIRKbb28oTId/7QkS6hizriQB8t6cgcR910kvcPuolIm3F+jG3XUR+F5Fzi1hE6P5aTUTeEJGN9r8R3swx7K9nicivdiM3TUTaedJesj/P/5cjIjsLWOZDRWSOvf/OEZFDPWmXisg6EVkpIj09nze35+u8hL0I22dYxarRBfA8gFlxLO8xAIvjVNblAPoD6A7gEAAVADzrSR8LYAWAugDOAPCwd+VFWpaI1AdwFYBmAF4E8Ij9eRqAJwDcFKflibe1AB4EMDpO5e22yxoWp/LyzQcwGMDc0AQR6Q3gdgAnAWgCax3c58nyLoCfAdQEcBeA8SJS22c+IwC0tMvpCeA2cc/I7gDwBoCmAPrkN7IichGAFao6O4blSzTuoyV3HwXg1PNjAJ8BqAHgWgBjRaRVDMU+BaAigCwA3QD0F5ErYqxnSwBvw/rBWQ3ApwA+sesPVR1kN/yZdiP4LoD3fcoqB2uZxwKoDmv/+1hEytnlPQrgcABDYG4zIwHcrKq5sSyLV7FpdEXkYgDZACbFqbxjAHQA8Ho8ygNwFoDXVPUvVd0F62BxkYhUFJFMAD0APKSqB1R1PoDxAK4salkAGgP4WVV3APgG1o4NWDvyJ6q6Mk7LE1eq+oGqfgRgS5zKm6mqbwFYHo/yPOU+r6qTAOwLkzwA1npZqKrbADwAYCBgdSnB2invVdW9qjoBwC8AzveZ1QAAD6jqNlVdDOCV/LJgNbbfqup2WA1YM/ss6nYAd8ZhMROC+2jJ3kc92sD6IfGUquaq6rcAfoT1IyNaZ8HqzdhjL/9r8P/bRqo3gCmqOlVVD8JaBw0AnBCaUaxu7vNhNabh9ID12OOnVTVHVUcCEAAnwvoRvUZV18GzPkXkAvvzGTEuh6FYNLr2Aed+ALeESWssItki0rgI5aUCeA7Wr5Z4Pv1DQuIMWGcz4pPeIYqyfgfQUUSqAegFYKGINAJwMYD/xFT7JLLX4bHJrkch2sM6E843H0BdEalppy1X1Z0h6e1DCxGR6gDqhykrP++vAE6213EXAAthNfBPq2p2nJYlrriPlvp91PhbRLm/FuVvG22ZfuWeD2ATgB98ymkPYIGaT4NaYH++CUBNEWkI4GRY67MygOGweqXiqlg0urAOOK+p6urQBFVdparVVLUo4yFDAcxQ1TlxqyEwEcDVYo3rVQXwL/vzivaB+EcAd4tIeRE5HNZGUDGKsrYAeAjAt7C6wG4F8Iyd51wR+d4el2oYx2VLOHsdFvcLgTIBbPdM58eVw6Tlp1f2Kcf7/dC8jwA4DsD3AF4AUA5AJwCfisg7IvKDiAyJdiEShPuoW1ZJ30eXAtgIYJiIpIvIKbDOHp2/RRT760QAt4tIZRFpAess1+9vG6lvAJwg1kVz5WD1ApXzKXcAgDdDGlUv3/1XVfMAXAer5+NWANfAGlZ6FkAnsa7j+FJE4vEjIvlvGbIHs3sBOCxO5R0Ca4fuUoTv7PJMtvPJNhpAIwCTYf3dnoDVpZJ/EOoHa7zrL1hdomMR5iwokrJU9V1Y4xMQkTMA5MAaS8w/Wzob1i/qiyNdxtJORL6A1ZABwD9U9e0oitkFoIpnOj/eGSYtPz3chRu7POn7QvOq6lYAF9n1ToH163wQrO7lX2F1Q88VkUl213RScR8tXfuoqh4QkT6wGpV/AZgN4L+wliFaQ+3yfoM1xPQugEv8Mkeyv6rqEhEZAKtHpD6s9bUI7vrML6sxrO7jawqoX4H7rz3kNMkurzOArrCuJ1kJ4FhY28KrAI4qYB4RSXqjC+uPlQVglYgA1i+SVBFpp6oFvXfPTzdYK2iRXV4FABVEZD2ABuEGxO1BeIeINAuTJw/AvfY/2L8O19j/oKp/AjjTU8Y7AGaGq2BhZXnKqADrir/TYHVr/aWqO0RkForx2F8yqOppcShmIYDOsA5AsOMNqrpFRBbCGnut7Oli7gzgnTB12SYi6+z0rz15F4aZ57UApqvqr2JdBfuUqu4XkV8AdET8LjKKRQ9wHy1V+6iqLoBnbFREpsF/PDSS8rbC+lGTX97D8Pnb2vkj2l9VdTysM1DY3flX4e8X8vUH8KOqFnT9x0IA/xQR8ZwNd4L1I8wh1gb5HIAbANQCkKqqf9rbZqdI6lyY4tDojgLwnmf6Vlg7+HVRlveF/f18FwG4FMA5sVyBJiI1YF31thxAWwBPArjf3jkhIm1h/QLLAXAhgFPsfEUuy2M4gDGqulZEFEBrEakL62rYuF5gFCv7CsA0AKmwDsjlARy0L4CIprwUWF1J6daklAeQp6r7Y6xnOVjDKgIg3S53v/23fxPAGBF5G9bV2MMBjAGs2wREZB6Ae0VkOKyDbCf4X0j1JoDhIjIb1tWy1wAwruYUkToArgdwtP3RCgA97QNgV1hnV8UB99FSsI96iUgnAMtg7QuDYf0IGhNDec1hXWSXDevvei3CXPAURbldAMyDdZX187AuVFsSku1yWBdZFWQygFwAQ0XkJbhnxd+G5LsawFxVnWcf0yqIdZtSY8RrfapqsfoH61aLsZ7pxrC6Bhr75B8D4MECyhsIYGqYzycDuNrnOz0ArA75rBWssZA9AP4EcEtI+k2wBuR3A5gKoGtI+i4Ax0VSlp2nDaxfdKmez4YB2Ayri6VjSP6VAHoleb1pyL8R4Za/sHXuWQeh5U0u7Hsh6QqgRZj1HlpuD0/6LQA2ANgB66raDE9alv39vfb66+VJ6wdgoWc6A1YX5Q67vHDr+E0AfT3TjQDMALANwJORbMdJXNfcR0vYPhpSl3/b29kuWD+CQveTou6vF8L6oboHViPZO5LvhaSH21+nwuoC3grgZQCVQtKPttdn5TDlfQHgTs/0YQDm2PvvXACHheSvBWt4p4rns34A1tvrrmek22eBf/tkr/w4bDyv2BvIHxHmbwnr19geAAN98hxvr5jscBtPcfoH655yCLVhAAAgAElEQVTSbLu+PZNdnyiXYbi942SH7lQFfGepvd5HF5BnH6yLJR5I9jLG+Pf52j7wTEp2XaKsP/fREr6PhixPmd5fI9k+C/rH9+kSEREFpLjcMkRERFTqsdElIiIKSKBXL5+c0pd92Unydd77UniuouH6TJ5ErE+A6zSZuI+WLn7rk2e6REREAWGjS0REFBA2ukRERAFho0tERBQQNrpEREQBYaNLREQUEDa6REREAWGjS0REFBA2ukRERAFho0tERBQQNrpEREQBYaNLREQUEDa6REREAWGjS0REFBA2ukRERAFho0tERBSQQF9iX1zk9jzciYeM+q+R9mLLFgmb786LjjKmq83b7NZp6e8Jmy8VTfblRxvTMx590YnbPT/YiRs/NtPIpwcPJrZipVxak0ZOXGdcthN/P6edka/NC25a7sKlia+YLbV2bWN6y2nusaL6uLlOrDk5gdWJSh6e6RIREQWEjS4REVFAymT38p+9M5y4RuquwOa7/oz9xvSB/u5vnhpnBlYNCiOtwSFO/MA9r/rmW3T9C0582sjjjDTduTP+FSvF0urVNabvnzzBiVun5znxiVvqGflyF/6W2Ip5eLuU+02da6QdVf5DJ77+l3+4CT8vTHi9SrLUWjWN6aVPNXbiHi3ddbvmhANGvtLSbc8zXSIiooCw0SUiIgoIG10iIqKAlJkxXUkv58QnnjgvKXWo/HN5Y/rCq7534u+qNTTScrO3B1Insmzs3cSJT6l4wDff4bMvcuLau5YltE6lUVrDBk5cddweI61TuVQnbv3NICduOcAcSw3S4geznPjCzIlG2uFP3+bEh/w8LagqlUgbhxzjxPfe+KaRdkbFr8J+p0+ts4zpg2vWxr9iScAzXSIiooCw0SUiIgpImele3nmu+xSqkQ2edeK2Hw0x8rXEjITVIae6GtNDqy9x4smV25qZ2b2cUCkVKxrTvYdOjeh7Ge9VdydU/TNSWNu6u0+d+ijred98bYdvdOIgn/OlR3c2pn8/82UnPuGXvkZao9Hu/pub2GqVSKmtmjvxq/982okPLWc2O3kIb92LlY3p+v9wbx07uG597BVMEp7pEhERBYSNLhERUUDY6BIREQWk1I7pavdDjennH3vGicfucG8PaTPcvO0jkWMzR5/yawJLp6LIOcYcQ3+wzmu+effkuY/vrPLO9ITVqTTyvjkIADads883b9f/3ODE9f4K7hYc7zju8Lff8M2363PzcZSVtixPWJ1Kg8W3u9c/eG8Hi9SMLu8Y08t+cvfD8966xUhr9tDPTpy3z38bKw54pktERBQQNrpEREQBKbXdy9vuMJ920zDNvfHglhvOcOL0bXMSWo+0+m6X1OuNzSfaHFD+5kmWFedF3t11wW99PFOl46k4QfnrmUxj+rduY5x4+EZzCKjB6+7beYK8BWdNj0pO3D3DvIGlw7QBTtz4WT51qiCp7VoZ09+c9LRnqoITPbbFHNqZne2+ZWhcc/MY6dXK81TBV/q9aKQ9NvocJ85b8WdE9U0WHvWJiIgCwkaXiIgoIKWqe3nLNUc78fsd/22kvbm9kxOnf5PYLmWvRfe7V28eULPTbMDKXk6cu3FTYHUi4Iwj5vumbc/ba0wfGOG+bD2F3ctFoirGtHcfmLEly0hL3bsRiZJS2Xy60dKH2jnxR2c/6cR5SDfyNe77S8LqVNps7ma+nD4rzX3q27V/He/Eq4/aZeRLqeQOBXYZ5F7Bfus1/zXy9avsbh/Hm++OwacTVjnxojOK95OreKZLREQUEDa6REREAWGjS0REFJBSNaab0mezEx+SlmGkvfbOqU7cEIm99D+1fWsnHnuS+5aSHDVfjr7qSfcS+0o5iXu7EVlyTj/CiZ9r8IpvvtUhr7VJ+f7n8BkpJv9r85ExfdXknk68amd9J97/mvkkqEitP859C9TpR84z0j455AXPlDuO233exUa+6vgtqnmXRbnmIRd5cP/+C17u6MQ18JOZb/duJ67/hHts/u9ZRxj5Lqn8mTuh5q1dG3LcMXvdlxN5pZOAZ7pEREQBYaNLREQUkBLdvZxau7YxPbzV5755Gz4c3NNklgyu5sRdM9xbJJ7f1s7IV2kCu5SDtOGI9MIzATjrs5uM6ZbgeopWnWcrGNPfjXLv9ehZwXww/WuNv3PiFLi3GuU9qYiGUQb8y3h3p3tLWM07I3vBOv1d5fPX+aZt7+12Idd4PbLy7mnyScgn/ueIU35u48Stts2MbAZJwjNdIiKigLDRJSIiCkiJ7l6WiuZjSXpX3O7E3WZdbqTVw+JA6gQAtbK2hv387RVdzXxYFjYfJUa5w7b5pi3e7z4Vp83IzUZakA/fL23SvjWf/vbMsSc68QPHZBlpq09xu4B/P+slJ56ZYz7V6rKvBkU075Zvulexfv7+aN98jy/q7cQN5i/0zUcF2zmhvvlBezcc2M4dovnhiG5Gtk2HuS/F0DPdY2eHdLObePEB9+6P9p6XHwDAh6c968T/OuoaN2H6gsIrHjCe6RIREQWEjS4REVFA2OgSEREFpESP6eZtzTamH9h0uBNf2ny2kfZD/eZOHO83T6Q1aWRM/3joe54p93fN3um1Qr7JMd1E23emO340+wjvi6/Nl9gvPVDHiXOX/ZHoapVZB9dvcOKKH2ww0lp94ManDzocflohsltCUjq5t5F4bx8CgAc3d3DiJje614KEPIyMiqDeJyuM6WV37HfiYTUXOfG/PjKvr/G7neuiP84wpvcOdW8RPffdyUbaFVX+cuI/hrrH3ObTC6l0EvBMl4iIKCBsdImIiAJSsruXd+40pr9a43YnTTn0HSNt3WdV3bSXj0ZRZbczu0Ays9wuqaMOWWnWy+c5NhLdg3UoBntrud3I6ZLqm++2Oec5cVMUv9sMqOhW3euu79AuzK8ecl+qnvlXMeyDLIFCh+2uHeY+2e31/zzpxK3SK5lf9Ly8oMVX7u0+bYYsMbLl7Xa7qB/99iwj7ao+7tDRY13dcYpXO5td1Hnzg7t11A/PdImIiALCRpeIiCggbHSJiIgCUqLHdENVv899LOQJIy4x0j7sMMaJH7vXfIlyJGbnmOOBuZ7fK13L7Q/JLQin8bO/GNN8g0ni5fTJDvu597GPANDw1cjeQETF1+ZrzWs1Fhz1vBOvPLjXSKuwKXSfpXjLfN999OMVuMWJt15o7nv7tmc4cdth7u16uZ6X24dqffsiY/qklu41GV+3n+DE995rnlc2OA9JxzNdIiKigLDRJSIiCkip6l7GTLf7turpZlL/HkOdOLtlBoqq5iv+XdJrPmhvTM85ckzYfKG3OFH8pbZqbkzPPmKsN9WJvtjVwciX/o35NhwqefacvMs37YJ5VxvTdb6bm+jqkIe3qznzff98kb7RK/RYuuNDz/7sORw/1mmCke+F+j2cON5PJowUz3SJiIgCwkaXiIgoIKWre7kAqZPd7qSak+Nb9t6Vlc0PjgyfT7sfakzLj/PiWxHChp51jGm/p1A9993JxnRLzAibj0qOl7u8ZUyvy3Wvkq35dMWgq0MBqv2y+xKMI0+71IlndDGfTHjjrVlO3Pyf7F4mIiIq1djoEhERBYSNLhERUUDKzJhuQoU8gCrF57cMx3ATb1+N8E8DA4A5Oe5TiNo+ttpI48vLS6bVdxzjxN0zzNuApue447ipvEWodMtzbzaq+YS73je/ZT6JbPHF7lPKznrnciNN5yxMUOVMPNMlIiIKCBtdIiKigLB7OR5CXk7v9xJ7Srw6J67xTftkx2FOnLtpcxDVoQTrd8kkJw59Uf1Vswc6cROYLxtJrVnDnahT0wlzF/8W3wpS4FK+/9mJe7wxzEhbdKXbvbzzIbPruUpf99bPRD49kGe6REREAWGjS0REFBA2ukRERAHhmG4c5JX3H8PdlJsTYE3KJslw3xp1ziHzffNt2Z/pxJrD9VLa5eW65xQbhxxjpJ1x9RQn/mh5fScuDi85p/hpMeovY/qtvvWc+IeO4420Uztf6cQpUxN3eyfPdImIiALCRpeIiCgg7F6Og7GnvmRML97vdjdfMuY2J26MaYHVqUzJdZ9GM2rxsUbSTcesdOLJf7Vw4gYI5ukzlDyLj3/difOON28nav+D25XYYsRuJ470JepUMhz8y3zy3H/PPcGJ+38zzkjbPGyfE9eZmrg68UyXiIgoIGx0iYiIAsLu5Ti4f8XZxvTuFxo4ceMJ7FJOND3ovq4g6/bdRlrbR/o7scyrDCpdvrzL7S5cdEd9I+2nGW2cuM0za4205uuXOnHuvn2gssH7xLGLlp9ipH162KtOfNVRg92E6QviWgee6RIREQWEjS4REVFA2OgSEREFhGO68XCSeVl6Jaz2yUiJlvv7CmO6cd8kVYQCUf7TmU686VMzrQWmO/FBEJn2nGveRjZj2iFOvK11JSeuPh1xxTNdIiKigLDRJSIiCgi7l4mIqMzJ3bzFmB7VqpkTV8dPCZsvz3SJiIgCwkaXiIgoIGx0iYiIAsJGl4iIKCBsdImIiALCRpeIiCggoqqF5yIiIqKY8UyXiIgoIGx0iYiIAsJGl4iIKCAlvtEVkTEisl9EVkaYv5WI7BKRXBG52idPDxHJs/OdGtcKx5mIXGXXU0WkRbLrEysRGSEiB+xlqlT4NwAR+cPeBsYWkEdFZLeIPBS/2iZGJMtTnJX1fbIgIpJhL8MBEXkw2fWJFPfLgonItyKyT0SmFpa3WDS6IjLZrvAu+9/SIhbxuKpmecrL3+l3ef6lAoCqLlPVTABTCilzrapmqupEu8w7Q8rbax8EatnpDUTkYxHZKiKrRWRQhMs+OrTBFJGnRWSbiPwkIg09n18qIiO931fV1+zlKTZEpK29EW4Xkd9F5NwiFjHO/tvvtsurJiJviMhG+98Ib2ZVbQ7g4QjK7ayqd3nqeZaI/Gqvz2ki0s6TliEiT4nIWntdvCAi6QUs84kiMldEdojIchG51pPWWUQWishmEbnF83m6iMwQkUZRLk/CiEgNEfnQPiD+KSKXFrGI0H0yw97Wd4jIeu/fIYZ9sr6IfGKvIxWRLG/mguZpp58kIktEZI+IfCciTfxmLCJZdp499nd6hZSzwp7HxZ7Pq9nbRGXPsubYy/p2IcuaECJysYgsttfrHyJyXBG+buyXdnmHi8gP9j60QURuzE+LYb8cJSJL7ePrwDDLcLP9t95hr98MT5rvegpTju/2ISKNRGS6fTx/IuR7X4hIV+9nqnoigIiO+cWi0bUNsVdopqq2jkN5j3vKy1TV3FgKU9WHveUBeAzAZFXdbGcZC2AFgLoAzgDwsIj0LKhMETkWQPOQz7oB6AKgHoCpAG63P68KYBiA4bEsR6KJSBqAjwF8BqAGgGsBjBWRVjEU+xSAigCyAHQD0F9Eroixni1hHfgGAagG4FMAn9j1B6y/e1cAHQC0AnA4fP72dmP8IYCXAVQFcBGAJ0Wks53lEQC3AugM4C4RqWd/fguACar6VyzLkiDPA9gPa3vuB+BFEWkfQ3kjALQE0ARATwC3SexnrHkAJgI4v6jzFOvH8gcA7oa1nc4GMK6Aeb0L4GcANQHcBWC8iNS2054GcBaA3gBeEPsHPqz1/qiq7oxm4eJNRE6Gddy6AkBlAMcDWB5DebVg/f1fhvV3aQHgq9hrivkABgOYG2aevWHtmyfBWq/NANznyVLQego1Av7b5B0A3gDQFECf/EZWRC4CsEJVZ0e7cMWp0S0xREQAXA5rpUBEMgH0APCQqh5Q1fkAxgO4soAy0gA8C+CGkKSmAKaqag6ASbA2KgB4CMC/VXVHHBclEdoAOATAU6qaq6rfAvgRQP8YyjwL1o+oPaq6EsBrKOBvG6HeAKao6lRVPQjrYNQAwAmeeY5U1a2qugnAyALmWQNAFQBvqWUWgMUA8s+cmwL4VlXXAPgNQGP7rOp8WD8oihWxug/PB3C3qu5S1akAPkFs63AAgAdUdZuqLgbwCoCBsdRTVTeo6gsAZkUxz/MALFTV91V1H6wDcGcRaRNaiP2D8XAA96rqXlWdAOAXuI19JVX91d7v9wOoaf94bqqq/41lGePsPgD3q+p0Vc1T1TX2NhmtWwB8qapv22fwO+2/c0xU9XlVnQRgX5jkAQBeU9WFqroNwAOw12kE6ylcWX7bR/4+ux3W9tVMRKrAavDvjGX5ilOj+4jd/fajiPTI/1BEGotItog0LmJ5g+2ugTki4vdHj9ZxAOoAmGBPS8j/+XGHAsq4GcAPqrog5POFAI4TkQqwfs0ttH9ltVbVd2KueXIYfwt7fR4bRRlhy4uxXt44tNzQ9IZ2j4NBVTfA+oV9hYikisjRsH4954/v/ArgFLGGCrIA/AHgGQDDVPVAHJYj3loBOKiqyzyfzQfQHij6Piki1QHUt8v4W3mJEME823vT7C7TP3zq1B7A8pAzVm9ZG8UaQugM6+x7G6z1OzQOixIX9tl3VwC1xRryWS0iz9nHmfw8Rd0vjwKwVayhmY0i8mkUx+miMtabHdcVkZoofD05Itg+fgVwsohUg9XzuBBWA/+0qmbHsgDFpdH9F6wzugYARgH4VESaA4CqrlLVaqq6qgjljYTVbVAHVvfRGBHpHsf6DgAwXlV32XXcCets7m4RKS8ih8P6dVUx3JfFGsP7B4B7QtNU9VdYjfl0AI0BPG4vz1ARGWqPn7xtbwzF0VIAGwEME2vM8hRYZ4/O38Jen4VecOAxEcDtIlJZrLHvK+Hzty2CbwCcINYFOuVg/Xot5yl3IoAbRaS23R2cfwD1m++7sNZnDqyxybs83ca3ArgO1tnizQC6A9gJYIVY1wF8LyJ9Y1yeeMoEENqjsh1Wl2Q0+2T+NQfbw5WXIIXNMzMkraA6FZZ3EKxGdhSs3oDrYG1f5UXkS3uM8QQkV10A6QAugHXScCiAw+AZMoliv2wI61h4I6xj1QpY+0Eiha6L/LhymLT8dL916v1+aN5HYP2dvgfwAqxjQydYbdM79nF4SDQLUCwaXVWdYXdN5KjqG7AasNNjKG+uqm5R1YOq+j9YY3fn+eUX8wKpAn+piUhFAH1hdy179IPVJfEXgBdhjfGu9inmaVjdPKEbSH79n1LVzqp6EYALAfwAa11dC+vsdzHssd7ixj5z6wNrXHs9gH8C+C/8/xaRGApgL6yu2Y9h7di+5dkXOuSvz34+9VwC64DxHIB1AGoBWOQp9yFYY0PzAEwD8BGAAwA2hJlfGwDvwRpyKAfr1/JtInKGPa8/VfV0VT3crv8DsBri/8AaRzwb1hhwjUj/IAm2C1Z3uVcVWD8Uoi0vv4yIyivKPhnlPIuyjAXmVdV5qtpDVY+EtQ1dCesColdhdeleAeAte1gqWfba/z+rquvsa1GeRAzHWbvMD1V1lt1Ffx+AY8L1BgGR7ZcRCF0X+fHOMGn56X7r1Pt9I689rHSRqnaG9YMqfyjwdlhnwb0ADBKRtkVdgGLR6IahMLv2ElpeyAVXhf16PxfAVgCTQ8r4U1XPVNXa9s5XC8BMnzJOAvBvsa6YW29/9pOEXCEqInVhNbT3w+r2XGA3arNg/eoqllR1gaqeoKo1VbU3rF4Mv79FJOVtVdV+qlpPVdvD2m59y1PV0zzr0/cqUVUdr6odVLUmgHthdf3OstP2quoQVW2gqs0AbAEwR1XzwhTVAcAyVf3SHitbCuBzAKeFyXsPgFfsLumOAGbbP75Ww7oQpThYBiDNvtgsX2dYXWxFZo+9rbPLiKi8Iu6T0cxzoTfNHsdu7lOnhbDG9LxnTH71fwrAcFXdC3f9roR1lul3QU/C2X+P1bCOhc7HMRa7oCjlRbpfFsJYb3a8QVW3oAjrqYjb5LUAptu9kPnrdD+s8eKORV2ApDe6Yl1W39vulk2zfwEdD6t7L9oyLxCRTBFJsbs3L4PVtRcPAwC8qWo+tFqs22Qqi0g5EbkMwCmwfkmG0wrWCj7U/gdYF+58GJLvSQAjVHUPrK6bIzwXbUV91WGiiUgne31WFJFbYY2djImhvOYiUtMeLz0N1k4Q8z2OItLFLrM2rK7BT+wz4PxbwA4Ry1Gwhinu9SnqZwAtxbptSOyhkTNhHZS882sHa929aH+0AsCJ9o+rlgCK3Lgkgj2++QGA+0Wkkj00cw6At2Io9k0Aw0Wkut0zcA1i2CbyiUh5APm3jGTY05HM80MAHUTkfPs798D6UbskdB722PY8APfa2/W5sH70TvDmE+vq4PKq+pn9Uf76bW/XcUusyxuj1wHcICJ17DHNm2HdZRBLeeeKyKFiXcF/N6yLQMP24EXKPoaWh3WilG7/zfPbqjcBXCUi7ewhtuGw12mk68mj0G1SROoAuB7WhXaAtU572sfhrojmOKyqSf0H69ffLFin9dmwxjJP9qQ3htUV0Njn+2MAPBjy2RRY/fM7YA2OXxzme5MBXO1TZg8Aq8N83gDAQQAtwqTdBGATgN2wLqDpGpK+C8BxPvPT0DIBnAjg85DPnoZ1kcZ0AA0LKyOJ6/Tfdj13AfgizLIV9LcYAWBsyGcXAlgLYA+snap3JN+L4G881d7utsK67aGSJ+14ACvteS4F0C/ku18AuDOkjr/a5a2GdTV0Ssh3vgNwpGe6M6zuyM0AbinK8gSwDmvA6lLfDevHwKWetGj2yQwAo+19ckPo8tp5otknNfRfpPOE1UW4BFY36WQAWZ60lwC85JnOsvPstbeHXmGWbx6AJp7PTrK3oXUIOQaF+xsFsE7TYY1PZsMa+hkJ60dCVPul/fl1ANbA2t8/BdCoKNsxwu+Xk8Os1x6e9Fvs9bkDVsOfEcl6gjUEuLCI2+SbAPp6phsBmGEv75MheQfC+tFR8HoIcqUnaEN6xd5Y/ogwf0t7o9sDYKBPnuPtlZaNMAf44vQP1nhRNqzL65sluz5xWJ7hsA702fA0goV8Z6m9DYwuIM8+WD/EHkj2MsZjeYrzv7K+TxayrBn2MuyGdWtL0usUYb3L/H5ZyLJ+DesH96TC8vLVfkRERAFJ+pguERFRWcFGl4iIKCBsdImIiAKSVniW+Dk5pS8HkJPk67z3435jPtdn8iRifQJcp8nEfbR08VufPNMlIiIKCBtdIiKigLDRJSIiCggbXSIiooCw0SUiIgoIG10iIqKAsNElIiIKCBtdIiKigAT6cAwiIip7UipWdOIu03YaaffWnufEpyw6z4nLnfxn4iuWBDzTJSIiCggbXSIiooCw0SUiIgoIx3QTIK1eXSfe3/KQiL6TvmyNMb30jmZOXG2R+9zsGov3GflSpvwcTRWJSox9Z3Uzpit8MdeJtWs7J15xdiUj33En/uLEU77t6Ft+/Z9ynbj8pzOjrieZvOO4y0a1duKPao8y8uV54r/m13fi5uCYLhEREcWAjS4REVFA2L0cpe2XHeXEW043u3xvP2yiE19e5X8Rlffa9sbG9HmVP3Ti6n3L+37vzAZdIiqfqLhLrVXTiXPHVXDi91o+aeTbkJvuxFVTJjtx47SK8DXgB9+kjZftceK1I8sZaf94+EYnrvnKT/7l098sv6uzEy/qOdKJ+y0/zci35aGmTtx84vTEVyzJeKZLREQUEDa6REREAWH3coiUzm2deMkN7tWQU0552shXO3WW+504/Ha5quqqkE/8u5SJSqNlz7hDLEvbvOZJMbuN66S68QvZrZx47k5ziGb17mq+80oV95rZz1t/GrZsABg3/N9OPGjxECMtZeo8kL/9dQ6G/XzBlJbGdNOJZavbnme6REREAWGjS0REFBA2ukRERAHhmG6I3U0rO/Gy0170pFT4e+YYvZTtPnXq7T+PiKqMqvg9XtUp9VIOdZ9etK+e+fSilX3cp35d0G2WkXZA3YG+795yn45U//vtRj79eWFc6llW6NGdjelxx7zsmXIPTRP3mmO6jw4b4MSVF252EzZtNfKlbPvLf94p7jpt9cRgJ1504bNGvubpmU68d/gOI63qQPfJcwfXb/CdV1mVnrnfiXfmuXHjr3OSUZ1ig2e6REREAWGjS0REFJBS272c1rCBMb34Xw2duO40tyuxyrvmE1BSctSJlx1wu0T+OmjeftAoLduJB/46wEjbtth9sk7dWW551aaZ3V26a5cTV81mN3E8aPdDjenl17vxO0e/4sRdyoXcGxKpYe4D8ffeut9IGpXtdl+/MP8EI63lVYudOG+f+QSzsupAVfPpT4eWcw9HeXD3m2GvX2nka/ThNCfORZTy3G+2uNk9BrQtZ94WtOCcZ5z4+47jjbTuvdxu6apj2b2c2qKpMb3w+NFOfOPak9x8381FWcYzXSIiooCw0SUiIgoIG10iIqKAlKox3dRqVZ242+crjLSPan3ixN1nm+M2XhlfuLeLDDtjoBPnLlxqzqut+yizGkv/MNJq5C0LW3b4h6JRNPKOdcduV7pDa/i8+/NGvuZp3lu93HHcr/eat4DduaiPE2evMsfvf+3j3kZy9wb37VKP15tt5OtcwX3p9pPdxhlpd9w80IkbPjINBOSWF9+0TtMGOnHjh4L7e7W8foYx/Vkv96XqfTO3GGnZZ+924qpjE1uvkmDpCP/HbgYp5zT39sudjfybuNpzzFvAdE4wt/zxTJeIiCggbHSJiIgCUqK7l1PKm2/iyRnvdi/fWetbI631B24fZJsP3W6Egm45CO1SNtIW/xZhLSkelr9j3gr0tu/tP2a38SUrTnbiWUvcWxra3LjYyFd7t7uua4fMe1CXXk68cWgTJ775RfO2o+F1JzvxlL31jbR5Q9wu6j5jz3Hig3+tRlnV+g7/7rzUOZV904J01yx32KFvz9eMtOvb/+DEn6F6YHUqrp46cpxv2o/vHO7E9RD7cMEfbx9mTD9z5LtO3LHcVCeum5rhW8bvB8wBv3PG3+zEzW+dHpo9bnimS0REFBA2ukRERAEpcd3LqdXdbpwlD7Qy0pa2fcGJ54Q8U9mkVj4AACAASURBVLvN/cudOHeHedUaFQ8plcyXEPx2f0cnXnyCeVVyiudK5Fmep4j1+/h6I1/r+9xu5FbZ7tXGeYhcx8prnPjrNLeLeva/uxj5aj7pXvnap1I2TP5X6pYlKZ3aOHGPal8bacsOuE/qqrXgQGB1Kkj17z1DWD2TV4/iKrVKFSeulGIedL/a6+7P9Z6KrEtZ0t2nlO3v2clIu+vF1534+PJzjLR0cY8HM3PcLuXLl/Q18t3S9CsnPrvSHiPthT7u8MHTo8914txF4e9GiRbPdImIiALCRpeIiCggbHSJiIgCUuLGdNde1taJl55rvnD6k93ueO9rZ55spOVuMp8aRcVP9tkdjelv+/7HiVNgvsh80l533ObRwe5bnlp8ZV7qH+lbaCTN3RVSWjc30l79qIYT//vNN5y4Y7mNIaW4dUwV8/dsxxmXOnGDjWV3W/xtgPvUooszNxlpxy7o78RV/jcLVPytuKmDEx9bfpKR1u67y524BX72LcP7dqKl19d14kUXPhsuOwBg0t5MY3rwlwOduM0zm504Y5m5rz0P9zqgZyc1MtI+a/OBEz/S2L39tNwi32pEhWe6REREAWGjS0REFJAS172888i9vmnPrHBflFxhWdntwiupNOS98vvU/zabnXnuk6fWH+neZrD3vG5GvhYt14X9/vZ95tPM+jZxX6x9fbW3jLTZ+93yu2d4bzYyu7y9ftxn3pTU4EF3WTQnJzR7mXHzaZ87sfcWIQAo93xNzxT335JAOvnffpn+RwXfNC/vixKW9HRvDQy9ra/f8tOceMdtDYy0lj+5t+tFOqT0+/J65gdtwueLN57pEhERBYSNLhERUUBKXPfyu91HeabM3wzj27kvtTz6yX8aaU0/2e/EqZPngoqf6h+bD8C/9vJ+Tjy2jfnC0rMruU+hOv8690lkuer/rKkcdR9wniEFbfpmmtml7DoY0pHVY8HFTlzjejNNlwfzrs6S5OUtxxvT5T+bmaSaULTa1NlQ5O9Il/bG9IfHvuiZSnei9pOvNfK1vMp9upzsm1/k+Rbmno3ue3jLT/7FiYvy9LpI8EyXiIgoIGx0iYiIAsJGl4iIKCAlbky3W4bb539AzXGz6inubSBLLjLfSnPgQjdvh0mDnLjqLPPWkV0N3bHCKu6LiVBrwW7fOm3uZL4dp+5k90lFubx1KWJ5O3ca0xmnuNPX1j3PSFs8IsuJT+nijr8s217HyPfnmlpOnFrO3QbObr3AyPd4vdkoqnbfmWNOrf/pvo3o4IbQp1WVTanVqhrTlVNWJ6kmlAgNK7pv00oJPYcTRTjLhpovlm+b7h7Tu8y6zImb9zOfYhXvsdX0zP3G9O6Dbr3y9u0LzR43PNMlIiIKCBtdIiKigJS47uWmn17jxMvOfCni73lfcry01ytuQq+4VMsw83b36UM3LfLcRnJmfF+GXJbkhnTXtrrOnV7p+bwc/jTytQyZzvfVh+2M6YK6l1cedF923efZ29yynzZvcck9eBBkWn2VeXtIv8rfOfHc3VkB16bock7f7pu2J6+cb1pZkafueVteaAewzxPl6tfNNqa932tX270FaVsc6hfK+3KFhcePNtKOX3ChE1dJ4BPReKZLREQUEDa6REREAWGjS0REFJASN6bb+nr3MvLe75u3bFz+3KdOXDHFfJPLmRXdF2Z7x3cToVuGe6n81MPeduL2/x5q5Gs+7KeE1oNMKx4+2onnHvFUSKr/+NwFj7vjuIc8P82Jw98QQSXZwRO7GNPvHfacZ8q81eXDx9y3mlXF9ERWq1SpdpV5O86MKe4tQ881do/hRz92q5Gv1Uj3+oyDa9ZGNe+249wyNuSab6wr/0wNzxTHdImIiEo8NrpEREQBKXHdy+q5LSP9mzlG2rttDvH93sgL3Ft3ctPdS9mPudW87ePRerNiraLB+5SWhp3Dv1CdEmftsGOc+Mt+jztxBfF/Af0z21oY0/Ven+fE8X4qDiWft0t5643mk+fapLtdyoPXdDfSqo1z31ZWVoYavLfcAMDxVb8tchmhXcOP9erjxJ0nuI8B/PWykUa+wSf0dOJ1Z9Qw0nK3bHXi7P7uMNKxN80w8t1T90cn7vKe2X3dfGIwQwQ80yUiIgoIG10iIqKAlLju5WhVGj8j7Oefdj7amH60v9u9vEfdB2J3+eE6I1+TV90roDcP3WOkzT7CfOE6BefAKV2N6Y+GuF3KjdP8u5RXeZ469cm/TjLSMvbEd8ihLKmy0nwpiffpXskkae6hL/tm98Uasw9/z8j39d4KTrzsbvPpWuUOFP0lGSVd7u8rjOn31ndz4nObTzTSmhy7yolTq1Rxy9ixw8h3cPlKJ55zmHseeHx/826PGgvcJ1lJrQNG2ornGjnxwuPdK85Dr1D2dik3vzU5V5zzTJeIiCggbHSJiIgCwkaXiIgoIGVmTNdP4y/NJ1ehvxtWFPcpRYtPeM3M1uRkJ/5f1pchpYb/LbNqvXmZe0vj/TgUDyvPNJ82luUzjrsu1xxbvPymfzpxxc/Dj/9T0VWaYP4tJz7Q1ombl99kpP3WsIMTH1y9JuZ55x17qBOvGGymnd/WvQ3s4TrmOK7Xw7cOcOIKX870zVdW7bvaHat9ckIbI+2zNh878Y2T3NutZr5kXkeTuTb827k2HWHeoHfEUPd2oicOmWqkeW/NHLU9y4nH/OdMI1/z0cl/CiDPdImIiALCRpeIiCggZb57OX32b8b0UXMvceLph7/r+723sr72TJm/XXLUvZz9TM9L7NsMNR+ibd5MQdFKrel22/983tMhqRkIp8fUIcZ08w/ZpRy0wdXM2082fOZ2Vc7e2jjm8h9tOsqJDy3nf6ibs9/dE/vPvMpIa/7tEifm/vp3ucvcY9oP55i3VFX/3H2611OHTHET7p8CP95u4rwiPP+tw9QrnLjFLZuduMaa5Hcnh+KZLhERUUDY6BIREQWEjS4REVFAyvyYbt7OncZ0vRuqO/FZo8924juzPjfyHZ3hjvBM2FXLSLvrfxc5cYub3UeNcUwoflKru+vpphnuGFGmhB/DBYDHtri3q7S8xhzL59uDguG9hWPjjT8YaffVnu9OeOOouYe3gyF733z3Ca+4bJz7uMGmt5tjgNxnI+d9nCMAfNTDvQVs5BXum4R2NzUf4fjlqe51GL2/vMlNKODVTa1f3WdMZ81a4NYjksomEc90iYiIAsJGl4iIKCBlvns51MGV7psxcKIbDh1qPtJm5xHu2yvaDN9spLX4MzlvryhLNp/tPv3mlIrfOXFuAV1S/7uvhxNX2s1bhJKhhueJQLN+aGWkPfmR22V4S3Wz+z8abb6/0onL/WI+mazhI9OcuCmK320lpUHuho1O3ODRjb75boD7tKpWiOyNXgXs5sUez3SJiIgCwkaXiIgoIOxejlDdkdPMaU9c3K+WK43Ov/UbJ85V/2uPW3w6yIlbTWCXcnES+kL0bzpUdmMcHnP5zTCv8ExEAeOZLhERUUDY6BIREQWEjS4REVFAOKZLJVLnCu6tXani/nacvs98hlC7x91bFTj2TkTJxjNdIiKigLDRJSIiCgi7l6lEuult92XjS655wYmvHH2Dka/RcvNWLyKiZOKZLhERUUDY6BIREQWEjS4REVFAOKZLJVKTe92x2t73HurEjcAxXCIqvnimS0REFBA2ukRERAER1ZL8OmAiIqKSg2e6REREAWGjS0REFBA2ukRERAFho0tERBSQEtfoisgIETkgIrtEpFKE3/lDRPaLyNgC8qiI7BaRh+JX2+CJyLcisk9Epia7LpESkTH2+lkZYf5W9vrPFZGrffL0EJE8O9+pca1wgEQkw16GAyLyYLLrEwnuowUrifuoV1nfXyNZnoIkpdEVkbb2hrddRH4XkXOLWMQ4Vc1U1d12edVE5A0R2Wj/G+HNrKrNATwcQbmdVfUuTz1HichSe2MYGGY5bhaR9SKyQ0RGi0iGJy1LRL4TkT0iskREevnN1D6wjrbLWS8it3jSGonIdBHZKiJPhHzvCxHpGrKsJwIYFMGyxpWI1BCRD+2D4p8icmkRi3hcVbM85fn+TVR1mapmAphSSJlr7e1kol1mfRH5RETW2gfwLG/mguZpp59kr8s99rpt4jfjgta/Xc4Kex4Xez6vJiJzRaSyZ1lz7GV9u5BljSsRGSIis0UkR0TGRFFE6D7a0/57bA93sOY+GiwRmWw3/Lvsf0uLWETo/prfEO/y/EsFYtpfRUTuEpFV9t/9PRGp4plnAxH52P67rxaRAv+mInKDvd/tsLftYz1pl4rIOhFZKSI9PZ83F5Fp+ctSxOUJK/BGV0TSAHwM4DMANQBcC2CsiLSKodinAFQEkAWgG4D+InJFjFUFgPkABgOYG5ogIr0B3A7gJABNADQDcJ8ny7sAfgZQE8BdAMaLSG2f+YwA/t/enUdZUZxtAH9etoEBRMAAIvsmgmwCIhoQUOISIRAxiqCiIiFBTVyIRlEQTTSfJ24IahSMuEQlOiKeuAUhxgURZBdIZFMQgSC7bMO83x/VU911nTvctWaceX7nzDlvT9Wt28vtW7erurrQOiinL4DfSfhr7/cAngHQHMCgwhNYRC4GsE5VF6SxfZk0GcAhAPUBDAPwmIi0T6O8CYi/T1JVAOAtABcm+54ichyAVwHcAfO5XQDgpWLeq7jj/xCAAQDOATAlckLfC+A+Vd2TysZl2NcA7gEwLUPl7QvKGpuh8grxHE3dtUElV0NVT8xAef8XKa+Gqh5Js7zLAVwG4AwADQFUAzApkv4cgHUw3zk/BfDHaIUZJSI9ANwHYAiAWgCmAsgTkYpBnXQfgFMAXBvzHo8AuCED22KVxJVuW5gd+KCqHlHV9wB8CLNzUzUA5oB/p6rrYXboVemuqKpOVtXZAA4UkXwFgKmqukJVdwC4G8AIwDQ/wBzA8aq6X1VfAbAM8b/srwBwt6ruUNWVAJ4sLAvmRH5PVXcB+BRAi+DX3q0Abkt3GzNBTBPihQDuUNW9qvoBgNeR3jEtbp+kRFW3qOoUmP2Y7Hv+HMAKVZ2hqgdgvoQ7iUjb2EISOP7VVXW5qi6B+aFSV0ROBdBcVV9OZxszRVVfVdXXAGzPUHnzVfVZAGszUV6kXJ6jZdcAmOP3laruBfAnABeLSK6I1ADQB8AfVPVwcC79HfG/95vBnL8L1TycYjqA4wDUg/nRtUlVNwP4J8yPM4jIkOD/n2Ryo0pLn64AONkuiOyMXvonUUaR5WVJe5hf2YWWAKgvInWDtLUxVyxLgv87RKQ2gOOLKKsw73IA/UXkWABdAayA+fJ4SFV3Zmhb0tUGQL6q/ifyP7sNItIkOKZNEiksgX2ScQm8p3O8g2bTNXHW6WjHf6uIdBKRTjBX3zsAPAzg+gxsihcpnqO+8Rwt3r0i8j8R+VBE+hT+M9nzNeLXQVPvQhGJ9+MlWbHf6zkwLQ4SJz3e9/6bACqKSI+gZekqAIsBfANgG8wP30YA+gNYIaaLZxxMK0ZGlUSluxrAVgBjRaSyiPwEwJkwzcMAAFU9NrhaStRbAG4VkZoi0gpmh+Ye5TXpqgFgV2S5MK5ZRFphek18X42Y18fmvRdALwD/AjAFQBUAHQHMEpEXROR9Ebk21Y3IkBoAdsf8z26Dqn4ZHNMvkyivsIzvlZclR3vPZI9pcXlHw1Syf4FpDfgVzC/sqiLydtDPeGYqG+FLCudoSeA5Gt8tMFd0J8B8DmeJSEsgpfMVMM2wrWGuHO8A8FcROSPNdXwLwEgxfe+1gnUGgNzgx9KHAO4QkaoicgpMK0W87/09AF4B8AGAgwDGAxilRgHMOfh3ADcDuAamG2ISgI7B+fi2iGTkQs77LEOqelhEBsFs0C0wfWMvw+yIVF0flPdfmOawvwEYGi+ziLwJc5IAwC9VNZWbVPYCOCayXBjvKSKtML2ovrq9kfQDsXlV9VsAFwfrXQHA+zBf2rfC/MIeAeAzEZkdNHuVhGS2N9HyCsv43j4piojsjSy2y8J7JntM4+ZV1cUwTWMQkeMB/BlAT5gv7d/C9Ke+LyJNtZw+p5XnaHbFNJk+IyJDAZwPtz8zmfKifer/EJHnYbpkPiwqf4Ln6zQAjQHMhamr/gzT5LwxSB8Gcy/JVzDdFs8hfmvY1QCuDNK/APATAG+ISBdV/TroopgdrFsnAN1g7j9YD+DHwXo8BeC0OOUnrESal1V1qaqeqap1VfUcmF9c89Mo71tVHaaqDVS1Pcx2xS1PVc+LdPanelfoCgCdIsudAGxR1e1BWguJ3IUapK8oYl12ANhcRFnfywtz09k8VV0OoAOABap6CKYvqkOK25EJ/wFQSURaR/4XbxuOKsl9Uvia6A0cyfxCT/Q9neMd9GO3jLNOCR9/mJsAx6nqfoTHdD2AygDi3dRT5vEc9U7hNtVmtbxEzldVLVDV8araTFUbwezvTcEfVHWDql6gqj9S1R4wfbTxvvc7A3hDzZ3HBWrukN4M4PRoJhERAI/CXMgdB6Ciqm6A6avvmPjmx1dSQ4Y6Bk0CuSJyM0x/yV/TKK+liNQN7kQ7D+aDn/aYRhGpIiJVYT48lYN1Ltxn0wFcLSLtgr6ccQi2IejbXAxgfPCawTAH7JU4bzUdwDgRqR3cmHMNYvaHiNQDMAbmBh7A3LXXN7ihoBsyfINKMoL+zVcBTBSR6kGz0s8APJtGsUfdJ6kIjmfhsJGcYDmR98wDcLKIXBi85k4AS1V1Vex7JHr8RaQ/gKqq+kbwr3UA+om56zsHGbqJKRUiUinYzoowfWFVg7s8Uy2vQlBeZbMoVUWkSgbWk+doksQMTTun8JiKyDAAvWGac1Mtc4iI1AiO808ADIe5mTKd9awTfLeLiLQD8ACAiUFzcOHQ05rBZ2A4zNXrA3GK+xTAT0WkRVBef5h7UZbH5BsJ4LOgNWo7gGrBe/dFpo6fqnr/A3A/zM0je2E6uFvFpO8F0CvOaycAeC7mf7+AaZL7DuZEOieR18WkaxHrMTf4f/SvTyT9RgBbYPoznwaQE0lrFrx+P0w/9tmRtGEwd9IVLufANKXsDsq7sYj1mw7goshyYwCfBPvxgZi8IwB84PmY1gHwGszQkC8BXBpJaxIc0yZxXvtXAPfE/C+RfTIXwMg4ZfYBsDHOcXb+En1PAGcDWBUc07kAmkXSHgfweCLHP/JeiwE0jfzvLJjmrM0ALjnaPsry8ZxQxL6aEElP9hztU0R5c4/2uiKOHc/R9I/tj2AqoT0AdgKYB6B/JD2V8/XfMP3cu2FuMrukiNfNRRLnK0yluBrme31D7D6H6YrZBvOd8wGAbjHp9jMK86NsIsx30x4AKwFcFpP/OJhK+JiYz8E3MOdl30S3p9j9XxIHPc0PzLhgJ++EGXqRyGtWBwdgWjF5DgQfmrtLehvT3D/vBh+q2SW9Lkms85PB8VmTYP7WwfH/DsCIOHl6B1+mO1HEj7Afyl/wZb8z+MyPL+n1SXCdeY4Wv60/uHM0Zv3L9fmayPYU98f5dImIiDwpLeN0iYiIyjxWukRERJ54Hafbv8JFbMsuIe8WzMjkcAAAPJ4lKRvHE+AxLUk8R8uWeMeTV7pERESesNIlIiLyhJUuERGRJ6x0iYiIPGGlS0RE5AkrXSIiIk9Y6RIREXnCSpeIiMgTVrpERESesNIlIiLyhJUuERGRJ6x0iYiIPGGlS0RE5InXWYaIiDLpiwdPs/Gaix930i7f0NvGW3ru9rZOlJz8fl1tvG5wWCXddNY/nHyjaq23cQW4E/gUIJxMafzWLjaetf5kJ1/DeyuGC/OXpbS+6eKVLhERkSesdImIiDxh8zKVaZUa1LfxrjOa2XhTf3du73UD/2Ljw3rESTtj8SU23vZVbRu3u+8bJ1/++i/TWldK3hmnfR43bXrT923ca/AvnbTcvE+ytk7l1aZbTneW97U+ZOOhXefHfd1d9cJzrwAFNq4Qc00YTTtp7ignrd7rOTau+dI8GzdE/M9HSeGVLhERkSesdImIiDxh8zL94ElO2LS09q5TnLRHhzxl4zOrfRe3jMMa/v6MNmMBwL87vxAudI6Eda9y8jW5KKHVpQyKNiEX5+ve7t2urfKysTbl25LrH3WWo3cUbzmy38ZTtrvN0G3eDJv+q/+3io2r/s/tAqo79WMbt8Si9Fa2BPFKl4iIyBNWukRERJ6w0iUiIvKEfboxjvQJ+wQr3bnFxrNOfN3JV1nCJ5sUN8Sk7u2VbSzrNzn5tg9oZ+M6ry130gr27Elmtcu1L8eGT7RZdtnDKZVx5YazbDy16bsJvWbx6dOc5YHontJ7U/a1umHe0TNRWnovG+Isv9fhJRtH+3EXdnGv9dpgQXZXrJThlS4REZEnrHSJiIg8KZfNy9EhJnsGdnbSxt8bNhlGh5i4g0iAw5G72YsbYnLKHSNs3KmB+xtnZrPwFvvux17npNWf9FHRK08AAO3ZycbTrpqU9Os7Pn29s9z87s9s3PbBMU7aqp9NTrp8ovLm2GsOOctvzK5r40HHLrTx4pMudfIdWfnf7K5YKcMrXSIiIk9Y6RIREXnCSpeIiMiTctmne7BPBxu/99CjcfPN2V/Dxnfe4z7yr/J3Gpvd2t00/C1TJfLkwd/d7A4x2VWQb+Mam91hR+SK9uECgN7zrY27hl303+t7z9tbz8bTRgy0cbNP3FlPtCDc/yfesMRJO++1X9n47sfDGVG65bjH7Ozl4TCvf55cM3YTKAtavjTaxrGT2EdFJ7sHOIQoG/K/2ugs35o3zMafDw+/Zw81cM+Niiuzu16lDa90iYiIPGGlS0RE5Em5aV6ONk/e+9gTcfMNXXO+jXePb2zj2nM+Lip7kWq1am7jzjPW2PikKu5vnLYzb7Bxm79zUu3ibO1e3Vn+tG3YVB99OtiuAnfYwviXw6eDNfs4sWOoBw86y5XfCZ+YM/ztsDlzxQC3a2JsnfBYP/m3K5y05kPdJmvKjOKalKmERSZ2qhBZ2N6+qpOtjnRFInIWhEOLjuzend66lSBe6RIREXnCSpeIiMiTctO8vOP2cBLl6N2u56/6uZOv4s3HhPGiz5CKnV3r23h8vZfj5mv8TkrFl0sVzt7uLEefAhZ9OtiVawc6+ZrdkXi3QCLa/Cq863nSj9s7aTfWWWXjYe0+ddI+QhUQlWWVGjdylu8b9LyNoxPaz/u9OylJhci1X/S8rhBzTdhn2UU2PjjDPfeiE9yXdrzSJSIi8oSVLhERkSesdImIiDwps326617s6Cyv6PK0jTfmh/27FW6v7eTTRUuTfq/orEUA0Oq3n4flR37XRCdKB4Bqr7lPRSJXpRMa2vimE/+Z0GvWzmjtLNfHtoyuU9S0mWc7yzdeuSpOTqKyKdqPe/7b7rC4gdV32Hj81i42nrX+ZCefzju2yLIHXvKBs3xji/A7YNDEnU5awcSwz/jcy0bZODrMCCgdQ414pUtEROQJK10iIiJPymzz8uXt3Kbb6K3oG/LDYUGYl3xzMuA2Ka9+yH0Y/8wm4aTn0Qfwb7j/RCdfLvgUquLs+HETGw+pMTNuvlFf9bHxCZEngAFAPkrGydXch7/Pb9HPxvlr13teG6Ls2Ns57AIaVcs9R3sv/YWNjzkvPC8b4nMkYuGf3GvCJY162XjcyKZO2mnnLrPxW8+Gk5JM3tnSyffmlWEZmL8MJYFXukRERJ6w0iUiIvKkzDYvZ1rF9m7T8Mrratl41YDJsdmt6Jy8NT9a56RxBt3ibTtFjp4JwJr7TrJxtW9Kxx3hF1R3n6D1QLcGNq7B5mXvOH9udlSdFZ5vF8xyJy44Bmtis6clf+MmGzeZsMlJ+3pCGHe55Tobx94BffdL4UQpv796tJNW6b2FGVjLo+OVLhERkSesdImIiDxhpUtERORJme3TfWVdZ2d5bN3w9vAuOfts3GvpgYTKOzX3VWe5b7XwdQWxmSNuWjLExo22rEjovcg4kht/xpGo0vJkr8pS0cbRmY+IyJ8T/vSRjZc839hJO/7tXTae+NSTTtpv/jDGxtmctYhXukRERJ6w0iUiIvKkzDYvNxju3lI+8LXBNn6jbfjklGizczJ6RW5LLxjqDg/5d+cXbFzvydyUyiegY8f1Ni4othG/dDis4SCwH8L6EpV10WFGADDjtnNsvHmCO4xsyrhHbHxF49/YuMmEj5BJvNIlIiLyhJUuERGRJ6x0iYiIPCmzfboFe/a4/zgrXO43+Nc23to1/u+O2ivDcR+1nnfb/7c9e9DGqzq/6KRN3dXMxrkrNtu4pGa8If825B9ylqttOxQnJxH5Um1mOLxwycL4w4kWX/OwjQdO6J7RdeCVLhERkSesdImIiDwps83LxcnNCyePb5aXWhmr+j1l49jhIZNXn2njhl8lNmEz/fCMHPRO3LSfPT3WWW4yJ7PDDsi4fENvG09v+n7cfF88eJqzzFmHKHY40SNL+tp49Jlrs/a+vNIlIiLyhJUuERGRJ+WyeTkVsZPYA+GEx7F3qtZ/pKqHNSr79t3Z0MYLnq7opHXLCZ/+9OWMDjZuclFqTxhLRfdq65zl+QfFxs3uX+Kk8flURKXMqR2cxWdPm2rjyTtbZu1teaVLRETkCStdIiIiT1jpEhERecI+3QStHV8lbtpFi0Y6yw3mfJbt1SkXKvxrkY3HPHStk/bpLZNs/G6Px2w8ou/1Tr6KGT4W617saOMzqi500k5fNNTGdfb9J6PvS6HvBvew8fSmT5TgmlDUhrtOd5ar/i+M608qHUPmKrZrY+PdE/c5aY0q7bfxWyN6RVIye58Ir3SJiIg8YaVLRETkCZuXi6E9O9n49R5TYlLDYUEyu7anNSq/jp/7rbPcrd9wGy/o/pyNN/Zxh2s1nZP+e++7MGzOfLlHONH1xwdznHx17uFQMR+a/25lSa8CBbZf3dPGy0ZOctJOmht2u9V3k9JWqXEjZ3nDpU2KzNfiT453PwAAAz1JREFUfPfJUrc1/puN5+13hwUNnhA+Ra7Opx+nu4px8UqXiIjIE1a6REREnrDSJSIi8oR9usXY2r26jZtXcvvrojMLVTqgoOwqWLrKWT7h9vCxnHl5dWz8+oj7nXznHnejjVuP+QTxSNf2Nt7Ss5aT9sRN4YTWJ1UJf6e2nTXKyddm3nxQ5kWHCAGJDxPqNeaXNm6Vx1mFsq2yuI9qXdknnIlt0brw+/LSj69x8kkk7t3iCxuv3lnPyTenwwwbV4A7FLAAGkkLS5yys7mTb+h74Wei3YTNTlqdjdnrx43ilS4REZEnrHSJiIg8YfNyMQ4cFzZZxE5U/9C37Wxc90k/zRIUOrJitY2fOTecfPqJv7jH6a0LHrDxy7262vjFF/o5+Z4aFY5p6JITf06gcz8fYuO2j+1x0jiTkH8tXxpt49iJ6XMRvzuBMqPu1PC77/R9o520rQMOFvmaZ3pOdZZPzQm/Z6Oz+xQ4Dc/uEKSC7e4TAlvkHS7yvaos/MJZbrN7gY3zi3xF9vFKl4iIyBNWukRERJ6webkYwwfFf5zRtJln27gZ2LxckvLXrrdxztAfOWmju/zGxpVv+cbGC6972MnXdtaYuOU3fzVsOM6Zs9TGBYcPJb2ulLzcPLeZ+Jy8zjZuBd6VXFrUfHFezHLR+SbilARLdLtvWmJRnHzxHUn6FdnHK10iIiJPWOkSERF5wkqXiIjIE/bpFuOVdWHf0di6mZ3ImLLjyLZtznLldyLL74ThQHR38rVBYk+T4rPHiCgdvNIlIiLyhJUuERGRJ2xeLobODh+kf1sj96Hr9ReUxpvRiYioNOOVLhERkSesdImIiDxhpUtEROQJ+3SLUf+Rj2y8/BE3rVqCQ0yIiIgK8UqXiIjIE1a6REREnogqn7FDRETkA690iYiIPGGlS0RE5AkrXSIiIk9Y6RIREXnCSpeIiMgTVrpERESesNIlIiLyhJUuERGRJ6x0iYiIPGGlS0RE5AkrXSIiIk9Y6RIREXnCSpeIiMgTVrpERESesNIlIiLyhJUuERGRJ6x0iYiIPGGlS0RE5AkrXSIiIk9Y6RIREXnCSpeIiMgTVrpERESesNIlIiLy5P8BcyFV9vNrQvcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    ckpt = tf.train.get_checkpoint_state('./')\n",
    "    if ckpt and ckpt.model_checkpoint_path:\n",
    "        saver.restore(sess, ckpt.model_checkpoint_path)\n",
    "        final_pred, acc = sess.run(\n",
    "            [pred, accuracy],\n",
    "            feed_dict={\n",
    "                x: mnist.test.images[:16],\n",
    "                y: mnist.test.labels[:16]\n",
    "            })\n",
    "        orders = np.argsort(final_pred)\n",
    "        plt.figure(figsize=(8, 8))\n",
    "        print(acc)\n",
    "        for idx in range(16):\n",
    "            order = orders[idx, :][-1]\n",
    "            prob = final_pred[idx, :][order]\n",
    "            plt.subplot(4, 4, idx + 1)\n",
    "            plt.axis('off')\n",
    "            plt.title('{}: [{}]-[{:.1f}%]'.format(mnist.test.labels[idx],\n",
    "                                                  order, prob * 100))\n",
    "            plt.imshow(mnist.test.images[idx].reshape((28, 28)))\n",
    "\n",
    "    else:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 结果分析\n",
    "\n",
    "通过增加L2正则项，改变学习率的下降方式为指数下降，增大训练次数后，准确率能保持在98.3%左右"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
